WO2021085572A1 - 無人移動体を用いたライブ映像配信方法、該ライブ映像配信方法において用いられる映像配信装置、及び、該映像配信装置が生成する映像データファイルを保存する映像アーカイブ装置 - Google Patents
無人移動体を用いたライブ映像配信方法、該ライブ映像配信方法において用いられる映像配信装置、及び、該映像配信装置が生成する映像データファイルを保存する映像アーカイブ装置 Download PDFInfo
- Publication number
- WO2021085572A1 WO2021085572A1 PCT/JP2020/040720 JP2020040720W WO2021085572A1 WO 2021085572 A1 WO2021085572 A1 WO 2021085572A1 JP 2020040720 W JP2020040720 W JP 2020040720W WO 2021085572 A1 WO2021085572 A1 WO 2021085572A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video distribution
- user
- live video
- live
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/765—Interface circuits between an apparatus for recording and another apparatus
- H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/20—Remote controls
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/14—Travel agencies
Definitions
- the present invention relates to a live video distribution method, in particular, a live video distribution method for simultaneously transmitting a live video signal generated based on the result of shooting a shooting target by an imaging means provided in an unmanned moving body to a plurality of user devices. Regarding. It also relates to a video distribution device used in the live video distribution method. Further, the present invention relates to a video archive device that stores a video data file generated by the video distribution device.
- tourism industry currently accounts for about 10% of the world's GDP, and is predicted to become the "largest industry in the 21st century" in the future. Under these trends, we are not satisfied with tourism in traditional tourist destinations, and there is an increasing need for tourism in unexplored regions, as well as in frontier spaces such as outer space and the deep sea.
- an aerial unmanned aerial vehicle is flown over an area such as a tourist spot, and the imaging means provided by the aerial unmanned aerial vehicle is based on the result of photographing an object to be photographed in the area.
- the live video signal generated in the above to a user device used by a user located at a location remote from the tourist spot, the user can experience a pseudo-sightseeing without going to the tourist spot.
- a live video distribution method has been proposed.
- a user at a remote location does not directly see the aerial unmanned aerial vehicle, but relies only on the image of the live video signal transmitted from the aerial unmanned aerial vehicle, and the user at hand. Since the configuration is such that the aerial unmanned aerial vehicle is remotely controlled by manipulating the equipment, the user is required to have a certain level of skill for remotely controlling the unmanned aerial vehicle, especially in the area where the aerial unmanned aerial vehicle flies. If is an area where the flight of unmanned aerial vehicles is restricted, users are required to have higher skills and a certain level of qualification to remotely control unmanned aerial vehicles. Therefore, there is a problem that a user who does not have the skill or qualification for remotely controlling such an unmanned aerial vehicle cannot enjoy the above-mentioned pseudo sightseeing.
- the present invention has been made in view of such circumstances, and even a user who does not have the skill or qualification for operating an unmanned aerial vehicle such as an unmanned aerial vehicle can move in the space near the object to be photographed. It is an object of the present invention to provide a live image distribution method in which an image of a live image signal transmitted from an unmanned moving object can be enjoyed.
- a plurality of users who are remote from each other can simultaneously observe an object existing in a tourist spot or the like in real time, receive an explanation from a guide, or feel as if they are together in the tourist spot or the like.
- the purpose is to provide a live video distribution method that enables dialogue and conversation.
- Another object of the present invention is to provide a video distribution device and a video archive device that allow a user who could not receive live distribution to enjoy the video captured by the imaging means of an unmanned moving object after the fact.
- the first aspect of the present invention relating to the live video distribution method is A live video signal obtained by shooting a shooting target is simultaneously transmitted from a video distribution device to a user device used by each of a plurality of users, and the video of the live video signal is displayed on a display means provided in the user device. It ’s a video distribution method.
- the live video signal is generated based on the result of shooting the shooting target by the imaging means included in the unmanned moving body moving in the space in the vicinity of the shooting target.
- the unmanned moving body moves based on a movement control signal generated based on the result of the operator other than the plurality of users operating the operator device. It was configured as follows.
- an unmanned vehicle or a walking robot that moves on the ground in the vicinity of the photographing target an unmanned aerial vehicle that flies over the vicinity of the photographing target, or the like can be used. it can.
- a flat display panel such as a liquid crystal display or an organic EL (ElectroLuminescence) display, or a head mounted display (hereinafter abbreviated as "HMD" can be used. ..
- the live video signal generated and transmitted by the unmanned moving object is abbreviated as "3D" by using a compound eye camera having a plurality of imaging units as an imaging means provided in the unmanned moving body.
- It can be a video signal.
- the stereoscopic HMD is used as the display means provided in the user device, and the user device is provided with a "sensor for detecting the movement of the user's head or eyeball" to project the left and right of the stereoscopic HMD. It can be configured to project "an image of a visual field range adapted to the movement of the user's head or eyeball" from the unit, and as a result, an immersive visual experience can be provided to the user.
- HMD means "stereoscopic HMD"
- the display means of the user device is an HMD
- the user device is referred to as an HMD. It is assumed that a 3D video signal that transmits both the image projected from the left projection unit and the image projected from the right projection unit is transmitted).
- the second aspect of the present invention relating to the live video distribution method is In the first mode of live video distribution method, All or part of the plurality of users are located at a location remote from the unmanned mobile body.
- the video distribution device receives a live video signal from the unmanned mobile body, performs necessary processing on the live video signal, and is used by a user located at the remote location via a public network. Send to the user's device, It was configured as follows.
- the video distribution device may be configured to directly receive a live video signal from an unmanned moving body, and may be a relay device such as a Wi-Fi router and /. Alternatively, it can be received via a public network.
- the Internet can be used as the public network.
- the live video signal is transmitted via the Internet, the video distribution of a general-purpose platform such as YouTube (registered trademark), Instagram (registered trademark), Vimeo (registered trademark) as (a part of) the video distribution device. You can use the server.
- the live video signal is transmitted from the unmanned moving body to the user device by any of the following routes (1) or (2).
- a relay device such as a Wi-Fi router.
- the network of the mobile communication system can be used as a part of the public network. Then, in that case, the video distribution device and / or the user device performs wireless communication with the mobile communication base station.
- the network of the 5th generation mobile communication system hereinafter abbreviated as "5G"
- 5G 5th generation mobile communication system
- the third aspect of the present invention relating to the live video distribution method is In the first or second mode of live video distribution method, The operator operates the operator's device while directly visually recognizing the unmanned moving body. It was configured as follows.
- the fourth aspect of the present invention relating to the live video distribution method is In the first or second mode of live video distribution method,
- the operator device receives a live video signal generated / transmitted by the unmanned moving object directly or via a video distribution device and / or a public network, and is provided in a display means provided with the driver device.
- the video of the live video signal is displayed,
- the operator is configured to operate the operator device while watching the image of the live video signal displayed on the display means included in the operator device.
- a flat display panel or an HMD can be used as the display means provided in the operator device.
- the left and right projection units of the HMD after providing a sensor that detects the movement of the user's head and eyeballs, the left and right projection units of the HMD "image of the visual field range adapted to the movement of the user's head and eyeballs". Can be configured to project.
- the network of the mobile communication system can be used as a part of the public network. Then, in that case, the operator device performs wireless communication with the mobile communication base station. At that time, in particular, by using a 5G network, a high-resolution live video signal can be transmitted to the operator's device with low delay.
- the fifth aspect of the present invention relating to the live video distribution method is In the live video distribution method of any one of the first to fourth modes,
- the operator is located at a location remote from the unmanned moving object,
- the video distribution device receives a movement control signal from the operator device, performs necessary processing on the movement control signal, and then transmits the movement control signal to the unmanned moving body via a public network. It was configured as follows.
- the network of the mobile communication system can be used as a part of the public network. Then, in that case, the video distribution device and / or the operator device performs wireless communication with the mobile communication base station. At that time, in particular, by using a 5G network, the movement control signal can be transmitted to the video distribution device with low delay.
- the sixth aspect of the present invention relating to the live video distribution method is In the live video distribution method of any one of the first to fifth modes, There are multiple unmanned moving objects, The plurality of unmanned moving bodies simultaneously shoot substantially the same shooting target from different viewpoints by the shooting means provided for each. "A plurality of live video signals having different display ranges" generated based on the result are simultaneously transmitted from the video distribution device to the user device. It was configured as follows.
- each of the user devices can receive only one live video signal of the "plurality of live video signals having different display ranges" at the same time. can do.
- each of the user devices may be configured to simultaneously receive a plurality of live video signals among the "plurality of live video signals having different display ranges".
- the display means of the user device may display the images of the received live video signals in parallel, or one of the received live video signals. It is also possible to display only the video of one live video signal. Then, in the last case, the video of the live video signal to be displayed can be switched by the user operating the user device.
- one operator may operate one unmanned moving object, or one operator may operate a plurality of unmanned moving objects. In the latter case, the risk of collision or crash can be reduced by adding the function of autonomous movement to the unmanned moving body.
- the seventh aspect of the present invention relating to the live video distribution method is In the sixth mode of live video distribution method, Each of the plurality of users receives a live video signal generated based on the result of the imaging means of which unmanned moving body among the plurality of unmanned moving bodies taken by the user device used by the user. "Do you want to do it?" Is selected, and the video of the selected live video signal is displayed on the display means provided in the user device. It was configured as follows.
- the user can be configured to select in advance before receiving the live video signal. Also, while receiving a live video signal generated based on the result taken by the imaging means of one unmanned moving object, the live video signal generated based on the result taken by the imaging means of another unmanned moving body is received. It can also be configured to switch to.
- the eighth aspect of the present invention relating to the live video distribution method is In the live video distribution method of any one of the first to seventh modes, There are multiple unmanned moving objects, The plurality of unmanned moving objects are deployed in a plurality of areas remote from each other. Each of the plurality of users said, "A live image generated based on the result of the user device used by the user device taking a picture of an unmanned moving object imaging means deployed in any of the plurality of areas. "Whether to receive a signal" is selected, and the video of the selected live video signal is displayed on the display means provided in the user device. It was configured as follows.
- the ninth aspect of the present invention relating to the live video distribution method is In the live video distribution method of any one of the first to eighth modes,
- the imaging means included in the unmanned moving object captures a wide-angle image including a panoramic image, an all-sky image, and an all-sky image.
- the display means of the user device cuts out and displays an image having a predetermined display range (hereinafter abbreviated as "predetermined display range image”) from the wide-angle image. It was configured as follows.
- the "panoramic image” is an "image that can be seen 360 degrees in the horizontal direction”
- the "omnidirectional image” is also called a “hemispherical image”, and is a “planetarium-like dome screen”.
- a “omnidirectional image” is a viewing angle (solid) that covers the entire field of view from the front, back, left, and right overhead to the feet.
- the "angle” is a 360-degree image
- the "wide-angle image” is a "normal field of view,” such as “panoramic image,” “omnidirectional (hemispherical) image,” and “omnidirectional image.”
- a flat display panel or an HMD can be used as the display means provided in the user device.
- the cutting out of the predetermined display range video from the wide-angle video may be performed by the video distribution device or the user device.
- the user device is provided with a "sensor for detecting the movement of the user's head and eyeballs", and the following (1) and (2) are provided. It can be done by any of the following methods.
- a wide-angle video signal is transmitted from the video distribution device to the user device, and based on the "result of the sensor detecting the movement of the user's head and eyeballs" in the user device, "the user's "Images in the visual field range adapted to the movements of the head and eyes” are cut out and projected from the left and right projection units of the HMD.
- the video distribution device From the user device to the image distribution device, "the movements of the user's head and eyes”
- a "signal based on the results detected by the sensor” is transmitted, and the video distribution device generates a video signal that transmits "a video in the visual field range adapted to the movement of the user's head and eyeballs" from the wide-angle video signal.
- the video signal is transmitted to the user device, and the user device projects a video based on the received video signal from the left and right projection units of the HMD.
- a plurality of audio signals such as a live operator, which is a collection of sounds emitted by a driver using the operator device or a person in the vicinity of the operator, can be transmitted directly from a video distribution device or via a public network. Simultaneously transmit to the user devices of the above, and simultaneously emit live sound by the sound emitting means of the plurality of user devices. It was configured as follows.
- the "sound collecting means for collecting the sound emitted by the operator who uses the operator device or a person in the vicinity of the operator" is the operator device. It is possible to provide a configuration provided with the above-mentioned video distribution device, a configuration provided with the above-mentioned video distribution device, or a configuration provided independently of those devices. Further, as the sound collecting means, a microphone can be used. In particular, in a configuration in which the operator device is provided with the sound collecting means, when a display means such as a flat display panel or an HMD is provided, the microphone can be integrated with the display means.
- a microspeaker can be used, and in particular, the configuration can be integrated with the display means such as the flat display panel and the HMD provided in the user device.
- the eleventh aspect of the present invention relating to the live video distribution method is described.
- All or a part of the plurality of user devices has a sound collecting means for collecting sound emitted by a user who uses the user device.
- All or part of the live user audio signal based on the result of sound collection by the sound collecting means of the user device and the superimposed audio signal superposed with the audio signal of the live operator or the like are directly or publicly transmitted from the video distribution device. Simultaneously transmit to the plurality of user devices via the network, and simultaneously emit live sound by the sound emitting means of the plurality of user devices. It was configured as follows.
- the video distribution device when there are four users A to D who use the user device, the video distribution device generates the following four types of superimposed audio signals, and the user
- the superimposed voice signals A, B, C, and D can be transmitted to each of A, B, C, and D (in the following, "the sound collecting means of the operator's device collects sound".
- “Live audio signal based on the result” and “live audio signal based on the result collected by the sound collecting means of the user device used by user A” are abbreviated as “operator audio signal” and "user A audio signal", respectively. To do).
- a video distribution device used in the live video distribution method of any one of the first to eleventh modes A live video signal is received from the unmanned mobile body, necessary processing is performed on the live video signal, and then the live video signal is simultaneously transmitted to the plurality of user devices directly or via a public network. It was configured as follows.
- the thirteenth aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- a movement control signal is received from the operator's device, necessary processing is performed on the movement control signal, and then the movement control signal is transmitted to the unmanned mobile body directly or via a public network. It was configured as follows.
- the fourteenth aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- a video distribution device used in any one of the fourth to eleventh modes of live video distribution method is received from the unmanned mobile body, necessary processing is performed on the live video signal, and then the live video signal is transmitted to the operator's device directly or via a public network. It was configured as follows.
- a video distribution device used in any one of the fifth to eleventh modes of live video distribution method is configured to receive a movement control signal from the operator device, perform necessary processing on the movement control signal, and then transmit the movement control signal to the unmanned moving body via a public network.
- the 16th aspect of the present invention relating to the video distribution device used in the live video distribution method in order to achieve the above object is described.
- a video distribution device used in any one of the sixth to eleventh modes of live video distribution method After receiving "live video signals with different imaging ranges" from each of the plurality of unmanned moving objects and performing necessary processing for each of the "live video signals with different imaging ranges", directly or to the public. Simultaneously transmitting to the plurality of user devices via the network.
- the seventeenth aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- a video distribution device used in a live video distribution method of any one of the seventh to eleventh modes An unmanned mobile body selection signal based on the selection result of the user who uses the user device is received from the user device, and the user device is used. Based on the unmanned moving body selection signal, it is determined to which user device the live video signal received from the plurality of unmanned moving bodies is transmitted. It was configured as follows.
- the eighteenth aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- a video distribution device used in any one of the eighth to eleventh modes of live video distribution method is received from the user device, and the area selection signal is received. Based on the area selection signal, it is determined to which user device the live video signal received from the plurality of unmanned moving objects is transmitted. It was configured as follows.
- a video distribution device used in the live video distribution method of any one of the ninth to eleventh modes A wide-angle video signal is received from the unmanned mobile body, necessary processing is performed on the wide-angle video signal, and then the wide-angle video signal is simultaneously transmitted to the plurality of user devices directly or via a public network. It was configured as follows.
- a predetermined range video signal for transmitting a predetermined display range video cut out from the wide-angle video is generated, and the predetermined range video signal is transmitted to a user device. It was configured as follows.
- the video signal in the predetermined display range of the user device can be transmitted from the video distribution device by any of the following methods (1), (2), and (3). .. (1)
- the user device is equipped with a "sensor that detects the movement of the user's head and eyeball", and then the user device sends a "signal based on the result detected by the sensor" to the video distribution device.
- a visual field range adapted to the movement of the user's head or eyeball is set, and a video signal in a display range corresponding to the visual field range is generated.
- the predetermined display range video signal is generated and the video signal is transmitted to the user device.
- the display range setting signal generated based on the result of the user operating the user device Based on the display range setting signal generated based on the result of the user operating the user device.
- the display range is determined, a video signal in the specified display range is generated, and the video signal in the predetermined display range is transmitted to the user device.
- the display range is defined regardless of the movement or the user's operation, the video signal of the defined display range is generated, and the predetermined display range video signal is transmitted to the user device.
- the video distribution device is a 3D video composed of video signals projected from each of the left and right projection units of the HMD. Send a signal.
- the 21st aspect of the present invention relating to a video distribution device used in a live video distribution method in order to achieve the above object is described.
- the live operator or the like voice signal is simultaneously transmitted to the plurality of user devices, either directly or via a public network. It was configured as follows.
- the 22nd aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- a video distribution device used in the eleventh mode of live video distribution method A live user voice signal is received from all or a part of the plurality of user devices, and a superimposed voice signal obtained by superimposing the voice signal of the live operator or the like on all or a part of the live user voice signal is generated. Simultaneously transmitting the superimposed audio signal to the plurality of user devices, either directly or via a public network. It was configured as follows.
- the 23rd aspect of the present invention relating to the video distribution device used in the live video distribution method is described.
- the video distribution device of any one of the twelfth to the twenty-second modes Recording of the video data transmitted by the live video signal is started substantially at the same time as the transmission of the live video signal is started, and the video data is saved as a video data file substantially at the same time as the transmission of the live video signal is completed. It was configured as follows.
- the video distribution device and the video archive device can be integrated.
- the live video distribution method of the present invention and the video distribution device used in the live video distribution method of the present invention it is only the operator who uses the operator device that controls the unmanned moving object, and other users. Can enjoy the video of the live video signal based on the result of shooting the imaged object by the imaging means of the unmanned moving body without maneuvering the unmanned moving body. Therefore, even a user who does not have the skill or qualification to remotely control the unmanned moving object can enjoy the image of the live video signal transmitted from the unmanned moving object moving in the space near the shooting target.
- the live video distribution method of the present invention and the video distribution device used in the live video distribution method of the present invention, "the result of photographing the imaged object by an imaging means of an unmanned moving body remotely controlled by one operator". Because multiple users can enjoy “video of live video signal based on”, the number of unmanned moving objects required is smaller than that of the conventional live video distribution method in which each user remotely controls an unmanned moving object. It is economical.
- the transmission of the live video signal from the unmanned moving body to the user device can be performed. Since it is performed via a public network such as the Internet, users can enjoy the video of the live video signal taken by the unmanned mobile even if they are in a place where the wireless signal from the unmanned mobile does not reach directly. Become. Therefore, for example, the user can enjoy the video of the live video signal of the autumn leaves of Arashiyama, Kyoto, and the video of the live video signal of the wild animals in Africa while staying in Tokyo.
- the operator can directly visually recognize the unmanned moving body and operate the device for the operator. Since it is maneuvered, it is possible to minimize the risk that the unmanned moving object collides with the imaging target or another unmanned moving object and damages the imaging target or the self or other unmanned moving object. Therefore, it is suitable for distributing a video of a live video signal of a precious archaeological site or a video of a live video signal of a precious exhibit of a museum or a museum.
- the operator can view the video of the live video signal captured by the unmanned moving body.
- the unmanned moving body In order to display the display on the display means of the operator's device used by himself / herself and operate the operator's device while watching the video of the live video signal, the unmanned moving body should be moved to a place where it cannot be sufficiently seen by the naked eye. Can be done.
- the movement control signal can be transmitted from the operator device to the unmanned moving body. Since it is performed via a public network such as the Internet, the operator can operate the unmanned mobile body even from a remote place where the radio signal cannot be directly delivered to the unmanned mobile body. So, for example, a pilot can fly an unmanned aerial vehicle over Kyoto Arashiyama or Africa while in Tokyo, while a user can enjoy live video signals of autumn leaves and wildlife while in New York. Will be able to.
- the fireworks launched at a fixed time are placed at different positions.
- the user can view the fireworks from "top view”, “bottom view”, “side view”, etc. , Can be enjoyed at the same time or while switching.
- the user controls each of a plurality of unmanned moving objects, for example.
- the live video distribution method of the eighth mode of the present invention by deploying the unmanned moving body to various tourist spots in Japan and abroad.
- the user can freely select a tourist spot and enjoy the live image of the tourist spot. This allows the user, for example, to enjoy live footage of the Parthenon in Greece and then live footage of the Egyptian pyramids while in Tokyo.
- the user moves the head or eyeballs or operates the user device.
- an image of the visual field range adapted to the movement of the user's head or eyeball can be cut out and displayed, or "an image of the display range that the user wants to display by the display means of the user's device" is displayed. You will be able to do it.
- the wide-angle video signal and the predetermined range video signal can be switched and transmitted according to the user device.
- a predetermined range video signal is transmitted to a user device whose display means is a flat display
- a spherical 3D video signal or an all-sky 3D video signal is transmitted to a user device whose display means is an HMD.
- the all-sky 3D video signal can be transmitted.
- only a predetermined range video signal is transmitted to the user device used by the user who uses the free service, and the user who uses the paid service uses it.
- An omnidirectional video signal or an omnidirectional video signal can be transmitted to the user device.
- the sound emitted by the operator or a navigator different from the operator is emitted by each. Sound can be emitted from the sound emitting means of the user's device used by the user. For this reason, even if the operator or navigator is remote from each user, he / she should explain in real time with his / her own voice about the object being photographed by the unmanned moving object, as if it were a tour conductor of a group tour. Can be done.
- the live video distribution method of the eleventh mode of the present invention and the video distribution device used in the live video distribution method of the mode, the result of shooting the shooting target by the same unmanned moving body imaging means Based on the live video signal, the superposed sound emitted by the user enjoying the image can be emitted from the sound emitting means of the user device used by each user. Therefore, even if each user is in a remote place from each other, they can share their impressions about the object being photographed by the unmanned moving object as if they were participating in the same group tour.
- the video data transmitted by the live video signal is the video data file. Therefore, even a user who could not enjoy the video of the live video signal in real time at the time of live distribution can enjoy the recorded video by playing the video data file.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the fifth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the sixth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. Is.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the eighth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the ninth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. Is.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the eleventh embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the twelfth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. Is.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the 20th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the twenty-first embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. Is.
- Is. A block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the 23rd embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.
- FIG. 24 It is an image diagram for demonstrating the screen displayed on the user apparatus used in the live video distribution method which concerns on Example 24 of this invention. It is an image diagram for demonstrating the screen displayed on the user apparatus used in the live video distribution method which concerns on Example 24 of this invention.
- FIG. 5 is an image diagram for explaining a screen displayed on a user device used in the live video distribution method according to the 25th embodiment of the present invention. It is explanatory drawing for demonstrating the imaging range, the viewpoint, the line of sight, and the viewpoint (center of the imaging range) of the imaging means.
- FIG. 1 is a block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the first embodiment of the present invention, and the function of the video distribution device used in the live video distribution method. Is.
- the live video distribution system of this embodiment includes the aerial photography drone 1, the driver's HMD_21 used by the operator, and the drone control controller 22 used by the driver (hereinafter, the driver's HMD_21 and the drone control controller 22). Together, it is referred to as "operator device 2"), the video distribution device 3, and the user A HMD_4A to the user D HMD_4D used by the users A to D.
- the HMD_21 for the operator and the HMD_4A for the user A to the HMD_4D for the user D all have a microphone and a microphone integrally configured.
- the drone maneuvering controller 22 has a wireless communication means, and the driver's HMD_21 and the drone maneuvering controller 22 are wiredly connected by a communication cable.
- the operator HMD_21 transmits and receives signals to and from the aerial photography drone 1 and the video distribution device 3 via the wireless communication means of the drone operation controller 22.
- the user A HMD_4A to the user D HMD_4D themselves have wireless communication means, and transmit and receive signals to and from the video distribution device 3 by their own wireless communication means.
- the aerial photography drone 1 is a multicopter type unmanned aerial vehicle, and by controlling the rotation speed of each rotor, it floats in the sky above the area where the object to be photographed, such as a tourist spot, and then moves straight and turns. Exercises such as exercise and rotational exercise can be performed.
- the aerial photography drone 1 is equipped with two spherical cameras that cover a viewing angle (solid angle) of 360 degrees as an imaging means, and photographs an object to be photographed in the area from the sky. Further, an omnidirectional 3D live video signal based on the results captured by the two omnidirectional cameras is generated, and the omnidirectional 3D live video signal is wirelessly transmitted to the drone control controller 22.
- the drone control controller 22 transmits the received spherical 3D live video signal to the operator's HMD_21 by wire via a communication cable, while wirelessly transmitting it to the video distribution device 3.
- the HMD_21 for the operator is equipped with a sensor that detects the movement of the driver's head and eyeballs, and the HMD_21 for the operator is left and right based on the received spherical 3D live video signal and the detection result of the sensor.
- the image of the live image signal is projected from the projection unit of. As a result, the operator can see the predetermined display range image of the spherical 3D live image signal corresponding to the movement of his / her head and eyeballs.
- the operator operates the drone operation controller 22 while observing a predetermined display range of the spherical 3D live video signal projected from the left and right projection units of the operator HMD_21.
- the drone control controller 22 is a radio-type joystick controller that generates a flight control signal based on a lever operation or a dial operation of the operator, and wirelessly transmits the flight control signal to the aerial photography drone 1.
- the rotation speed of the rotor blades of the aerial drone 1 is controlled based on the received flight control signal, and the aerial drone 1 flies as intended by the operator.
- each of the HMD_4A for the user A to the HMD_4D for the user D also has a sensor for detecting the movement of the head and the eyeball of the users A to D, and the HMD_4A for the user A to the HMD_4D for the user D of each sensor. Based on the detection result, a signal for setting the display range of the image projected from the left and right projection units (hereinafter abbreviated as "display range setting signal”) is generated, and the display range setting signal is wirelessly transmitted to the image distribution device 3. Send.
- the video distribution device 3 is based on the display range setting signals received from each of the user A HMD_4A to the user D HMD_4D, and from the all-sky 3D live video signal received from the drone control controller 22, the user A HMD_4A to the user.
- a 3D live video signal to be transmitted to each of the D HMD_4D is generated, and the 3D live video signal is wirelessly transmitted to each of the user A HMD_4A to the user D HMD_4D.
- Each of the user A HMD_4A to the user D HMD_4D projects an image from the left and right projection units based on the 3D live image signal received from the image distribution device 3.
- the users A to D can see the predetermined display range image of the 3D live image signal corresponding to the movement of their own head and eyeballs.
- the users A to D can enjoy the predetermined display range of the spherical 3D live video signal of the target captured by the spherical camera of the aerial drone 1 without performing any maneuvering by themselves.
- the video distribution device 3 is configured to cut out a predetermined display range video based on the display range setting signals received from each of the user A HMD_4A to the user D HMD_4D.
- a predetermined display range image is cut out based on the received all-sky 3D live video signal and the detection result of the sensor. You can also do it.
- the operator verbally explains the object photographed by the spherical camera of the aerial drone 1 while watching the image of the spherical 3D live video signal projected from the left and right projection units of the HMD_21 for the driver. To do.
- the sound emitted by the operator is collected by a microphone integrated with the HMD_21 for the operator, and wirelessly transmitted to the video distribution device 3 as an audio signal via the drone operation controller 22.
- the navigator wears an HMD with a microphone similar to the HMD_21 for the operator on the head, and verbally explains while watching the video of the live video signal projected from the left and right projection units on the HMD.
- the sound emitted by the navigator is collected by a microphone integrally provided in the HMD, and is wirelessly transmitted to the video distribution device 3 as a voice signal via the drone control controller 22.
- each of the users A to D is watching the predetermined display range image of the 3D live image signal projected from the left and right projection units of the HMD_4A for the user A to the HMD_4D for the user D, and the spherical image of the aerial image drone 1. Quoting impressions about the subject photographed by the camera.
- the audio emitted by each of the users A to D is collected by the microphones integrally provided in the HMD_4A for the user A to the HMD_4D for the user D, and is wirelessly transmitted to the video distribution device 3 as an audio signal.
- the video distribution device 3 generates the following four types of superimposed audio signals based on the audio signals generated and transmitted by the microphones integrally provided in the HMD_21 for the operator and the HMD_4A for the user A to the HMD_4D for the user D.
- Superimposed audio signals A to D are transmitted to each of HMD_4A for user A to HMD_4D for user D, respectively (in the following, "audio signals generated and transmitted by the microphone of HMD_21 for operator” and "HMD_4A for user A”.
- the "voice signal generated and transmitted by the microphone” is abbreviated as "driver voice signal” and "user A voice signal”, respectively).
- the video distribution device 3 generates the following superimposed audio signal 0 and transmits it to the HMD_21 for the operator.
- Superimposed audio signal 0 ?? Audio signal superimposing “User A audio signal + User B audio signal + User C audio signal + User D audio signal” HMD_21 for operator controls the superimposed audio signal 0 from the video distribution device 3 by drone. Received via the controller 22.
- each of the microspeakers integrally provided in the HMD_21 for the operator and the HMD_4A for the user A to the HMD_4D for the user D emits sound based on the received superimposed voice signal.
- each of the operator and the users A to D can hear the voice superimposed with the voice emitted by a person other than himself / herself.
- users A to D can hear the explanation given verbally by the operator (or navigator) and the impression dictated by their own user.
- the flight of the aerial photography drone 1 can be adjusted or the commentary can be supplemented according to the impressions.
- the video data and the audio data transmitted by those signals are temporarily stored in the storage means provided by the video distribution device 3.
- a video / audio data file in a format such as MP4 is created from the temporary data and stored in the storage means.
- the HMD_21 for the operator is configured to transmit and receive the all-sky 3D video signal and the audio signal via the drone operation controller 22, but the HMD_21 for the operator has a wireless communication means. After that, it can be configured to directly transmit and receive to and from the aerial photography drone 1 and the video distribution device 3.
- FIG. 2 is a block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the second embodiment of the present invention, and the function of the video distribution device used in the live video distribution method. Is.
- the spherical 3D live video signal is transmitted from the aerial drone 1 to the video distribution device 3 by direct wireless transmission, not via the drone control controller 22.
- FIG. 3 is a block diagram for explaining the configuration and function of a live video distribution system that realizes the live video distribution method according to the third embodiment of the present invention, and the function of the video distribution device used in the live video distribution method. Is.
- the signal transmission / reception between the aerial photography drone 1 and the drone control controller 22 is not performed by direct wireless communication, but is performed via the video distribution device 3.
- the video distribution device 3 wirelessly transmits the spherical 3D live video signal received from the aerial photography drone 1 to the drone control controller 22.
- the drone control controller 22 generates a flight control signal based on the lever operation and dial operation of the operator, and wirelessly transmits the flight control signal to the video distribution device 3.
- the video distribution device 3 wirelessly transmits the flight control signal received from the drone control controller 22 to the aerial photography drone 1 after performing necessary processing.
- the rotation speed of the rotor blades of the aerial drone 1 is controlled based on the received flight control signal, and the aerial drone 1 flies as intended by the operator.
- FIG. 4 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the fourth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the live video distribution system of this embodiment includes a video archiving device 8 in addition to the aerial photography drone 1, the HMD_21 for the operator, the controller 22 for the drone operation, the video distribution device 3, and the HMD_4A for the user A to the HMD_4D for the user D. ing.
- the functions of these components are the same as those of the first embodiment except for the points described up to paragraph [0107].
- the signal transmission / reception between the drone control controller 22 and the video distribution device 3 is performed not by direct wireless communication but via a public network. It is done through.
- the drone control controller 22 transmits the spherical 3D live video signal received from the aerial photography drone 1 to the 5G base station 5, which is a base station of the 5G network, and the spherical 3D live video signal is transmitted to the 5G base station 5. It is transmitted to the video distribution device 3 via a 5G network (not shown in the figure) and the Internet 6.
- the drone control controller 22 transmits a voice signal generated based on the result of sound collection by the microphone integrally configured with the driver HMD_21 to the 5G base station 5, and the voice signal is transmitted to the Internet 6 Is transmitted to the video distribution device 3 via the 5G network and the Internet 6.
- the superimposed audio signal 0 generated by the video distribution device 3 is transmitted to the drone control controller 22 via the route of the Internet 6 ⁇ 5G network ⁇ 5G base station 5, and is finally transmitted to the operator HMD_21.
- the video distribution device 3 writes the video data and the audio data transmitted by those signals as temporary data to the storage means provided by the video distribution device 3 substantially at the same time when the transmission of the 3D live video signal and the superimposed audio signal 0 is started. .. Further, substantially at the same time when the transmission of the 3D live video signal and the superimposed audio signal 0 is completed, a video / audio data file in a format such as MP4 is created from the temporary data, and the video data file is transmitted to the video archive device 8. .. The video archive device 8 saves the transmitted video data file.
- the drone control controller 22 has a configuration having a 5G communication function, but a 5G communication unit (such as a 5G compatible Wi-Fi router) is separately provided, and the drone control controller 22 has the 5G communication. It may be configured to communicate with the unit by wire or wirelessly.
- a 5G communication unit such as a 5G compatible Wi-Fi router
- the video archive device 8 is provided as a device separate from the video distribution device 3, but the video distribution device 3 itself stores the video data file (that is, the video distribution device 3 and the video archive). It may be configured (integrally configured with the device 8).
- the video archive device 8 is configured to store the video / audio data file transmitted from the video distribution device 3, but the video data is stored in the aerial photography drone 1.
- a recording device is provided to record the image taken by the imaging means in the image data recording device, and after the aerial photography drone 1 returns from the flight, the image data is collected as an image data file and the image data file is collected. It can be configured to be stored in the video archive device 8. Then, by adopting such a configuration, it is possible to store high-quality video and distribute the high-quality video of the saved video data file to the user device without being restricted by the capacity of wireless communication. it can.
- FIG. 5 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the fifth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the spherical 3D live video signal is transmitted from the aerial drone 1 to the video distribution device 3 without going through the drone control controller 22. That is, the aerial photography drone 1 transmits an all-sky 3D live video signal to a 5G base station 5, which is a base station of a 5G network, and the all-sky 3D live video signal is a 5G network (described in the figure). Not) and is transmitted to the video distribution device 3 via the Internet 6.
- FIG. 6 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the sixth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the transmission / reception of signals between the video distribution device 3 and the HMD_4A for user A to the HMD_4D for user D is not performed by direct wireless communication, but is performed via a public network.
- each of the HMD_4A for the user A to the HMD_4D for the user D outputs the display range setting signal generated based on the detection result of the sensor that detects the movement of the head and the eyeball of the user A to D to the ISP server A_7A to the ISP server. It is transmitted to the video distribution device 3 via each of D_7D and the Internet 6.
- the video distribution device 3 generates a 3D live video signal generated based on the display range setting signal received from each of the user A HMD_4A to the user D HMD_4D and the all-sky 3D live video signal received from the aerial photography drone 1. , It is transmitted to each of the HMD_4A for the user A to the HMD_4D for the user D via the Internet 6 and each of the ISP server A_7A to the ISP server D_7D.
- each of the HMD_4A for the user A to the HMD_4D for the user D transmits a voice signal generated based on the result of sound collection by the integrally configured microphone to each of the ISP server A_7A to the ISP server D_7D and the Internet 6 It is transmitted to the video distribution device 3 via and.
- the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to each of the user A HMD_4A to the user D HMD_4D via the Internet 6 and the ISP servers A_7A to ISP server D_7D.
- FIG. 7 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the seventh embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the transmission / reception of signals between the video distribution device 3 and the HMD_4A for user A to the HMD_4D for user D is not performed by direct wireless communication, but is performed via a public network.
- FIG. 8 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the eighth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the user A device to the user D device are composed of the user A smartphone_41A to the user D smartphone_41D in addition to the user A HMD_4A to the user D HMD_4D, and the user A smartphone_4A.
- the HMD_4D for the user D is wiredly connected to the smartphone for the user A_41A to the smartphone for the user D_41D by a communication cable, respectively.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the HMD_4A for the user A to the HMD_4D for the user D does not send and receive signals to and from each of the ISP server A_7A to the ISP server D_7D, but the smartphone for the user A_41A to the smartphone for the user D_41D.
- Signals are transmitted and received to and from each of the 5G base station A_5A to the 5G base station C_5C and the 5G base station D_5D.
- each of the HMD_4A for the user A to the HMD_4D for the user D outputs a display range setting signal generated based on the detection result of the sensor that detects the movement of the head and the eyeball of the users A to D to the smartphone_41A for the user A.
- the video distribution device 3 generates a 3D live video signal generated based on the display range setting signal received from each of the user A HMD_4A to the user D HMD_4D and the all-sky 3D live video signal received from the aerial photography drone 1. , Internet 6 and 5G networks, and 5G base stations A_5A to 5G base stations D_5D and user A smartphones_41A to user D smartphones_41D, and user A HMD_4A to user D HMD_4D. Send to each.
- each of the HMD_4A for the user A to the HMD_4D for the user D generates an audio signal based on the result of sound collection by the microphones integrally configured, and the smartphone for the user A_41A to the smartphone for the user D_ It is transmitted to the video distribution device 3 via each of the 41Ds, each of the 5G base stations A_5A to 5G base stations D_5D, and the 5G network and the Internet 6.
- the superimposed audio signals A to D generated by the video distribution device 3 are the Internet 6 and the 5G network, each of the 5G base stations A_5A to 5G base stations D_5D, and the smartphones for user A_41A to the smartphones for user D_41D. It is transmitted to HMD_4A for user A to HMD_4E for user D via each of the above.
- FIG. 9 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the ninth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the user A device to the user D device are composed of the user A smartphone_41A to the user D smartphone_41D in addition to the user A HMD_4A to the user D HMD_4D, and the user A smartphone_4A.
- the HMD_4D for the user D is wiredly connected to the smartphone for the user A_41A to the smartphone for the user D_41D by a communication cable, respectively.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the HMD_4A for the user A to the HMD_4D for the user D does not send and receive signals to and from each of the ISP server A_7A to the ISP server D_7D, but the smartphone for the user A_41A to the smartphone for the user D_41D.
- Signals are transmitted and received to and from each of the 5G base station A_5A to the 5G base station C_5C and the 5G base station D_5D.
- FIG. 10 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the tenth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the signal transmission / reception between the aerial photography drone 1 and the drone control controller 22 is not performed by direct wireless communication, but is performed via the video distribution device 3 and the public network. Further, the transmission / reception of the audio signal between the HMD_21 for the operator and the video distribution device 3 is performed by direct wireless communication without going through the public network.
- the video distribution device 3 wirelessly transmits the spherical 3D live video signal received from the aerial photography drone 1 via the 5G base station 5 ⁇ 5G network ⁇ Internet 6 to the drone control controller 22.
- the drone control controller 22 generates a flight control signal based on the lever operation and dial operation of the operator, and wirelessly transmits the flight control signal to the video distribution device 3.
- the video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller 22, and finally aerial photography drone 1 from the 5G base station 5 via the Internet 6 and the 5G network. Send to.
- the rotation speed of the rotor blades of the aerial drone 1 is controlled based on the received flight control signal, and the aerial drone 1 flies as intended by the operator.
- the drone control controller 22 wirelessly transmits an audio signal generated based on the result of sound collection by the microphone integrally configured with the driver's HMD_21 to the video distribution device 3.
- the superimposed audio signal 0 generated by the video distribution device 3 is transmitted to the operator HMD_21 via the drone operation controller 22.
- FIG. 11 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the eleventh embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the aerial photography drone 1 does not send and receive signals to and from only one 5G base station 5, but sends and receives signals to and from 5G base stations I_5I to 5G base stations III_5III. It is done by the method. In this embodiment, there are three 5G base stations, 5G base stations I_5I to 5G base stations III_5III, but the number of 5G base stations can be increased in principle.
- the aerial photography drone 1 is not limited to the range where wireless communication is possible with only one 5G base station, and can fly anywhere as long as cellular communication is possible. , It is possible to shoot an object to be photographed spread over a wide area.
- FIG. 12 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the twelfth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the transmission / reception of the signal between the drone control controller 22 and the video distribution device 3 is not performed by direct wireless communication, but is performed via a public network.
- the video distribution device 3 transmits the all-sky 3D live video signal received via the aerial photography drone 1 to 5G base stations I_5I to 5G base stations III_5III, 5G network and the Internet 6 via the Internet 6 and the 5G network. Finally, it is transmitted from the 5G base station 50 to the drone control controller 22.
- the drone control controller 22 transmits a flight control signal generated based on the operator's lever operation and dial operation to the 5G base station 50, and the flight control signal is transmitted to the video distribution device via the 5G network and the Internet 6. Sent to 3.
- the operator HMD_21 transmits a voice signal generated based on the result of sound collection by the integrally configured microphone to the 5G base station 50 via the drone control controller 22, and the voice is transmitted.
- the signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.
- the superimposed audio signal 0 generated by the video distribution device 3 is finally transmitted from the 5G base station 50 to the HMD_21 for the operator via the Internet 6 and the 5G network.
- FIG. 13 shows the configurations and functions of the live video distribution system and the local system that realize the live video distribution method according to the thirteenth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8, local system 9) It is the same as the first embodiment except for the points described up to.
- the live video distribution system of this embodiment includes aerial drone (1) _11 and aerial drone (2) _12 as components, and aerial drone (1) _11 is drone operation by the operator (1). Flying based on the flight control signal generated and transmitted based on the operation of the controller (1) _221, the aerial drone (2) _12 is the operation of the drone control controller (2) _222 by the operator (2). Fly based on the flight control signal generated and transmitted based on.
- the HMD_211 for the driver (1) and the HMD212 for the driver (2) are equipped with sensors that detect the movements of the head and eyeballs of the driver (1) and the driver (2), respectively.
- the HMD_211 for 1) projects the image of the live image signal from the left and right projection units based on the all-sky 3D live image signal received from the aerial photography drone (1) _11 and the detection result of the sensor, and the operator.
- the HMD_211 for (2) projects the image of the live image signal from the left and right projection units based on the all-sky 3D live image signal received from the aerial photography drone (2) _12 and the detection result of the sensor.
- the users A to C and the user D are displayed in the virtual space projected by the HMD_4A for the user A to the HMD_4C for the user C and on the display of the notebook PC_4D for the user D.
- aerial drone (1) _11 and aerial drone (2) _12 Based on the profile of 2) and the flight plan (flight path and flight path) of aerial drone (1) _11 and aerial drone (2) _12, aerial drone (1) _11 and aerial drone (2) Select which of _12 receives the video of the live video signal to be captured.
- the user A to the user C select the aerial photography drone (1) _11 and the user D selects the aerial photography drone (2) _12.
- the users A to C operate a different operating means (not shown in the figure) from the HMD_4A for the user A to the HMD_4C for the user C, so that the user D can use the user D.
- This is performed by operating the input means (keyboard or pointing device) of the notebook PC_4D for use, and an unmanned moving object selection signal based on the selection result is transmitted to the video distribution device 3.
- the video distribution device 3 includes the aerial photography drone (1) _11, the aerial photography drone (2) _12, the user A HMD_4A to the user C HMD_4C, and the user D notebook PC_4D based on the received unmanned moving object selection signal. Is associated with.
- the 3D live video signal generated from the spherical 3D live video signal received from the aerial photography drone (1) _11 is transmitted to the user A HMD_4A to the user C HMD_4C, and is used for the user D.
- a two-dimensional live video signal generated from the spherical 3D live video signal received from the aerial photography drone (2) _12 is transmitted to the notebook PC_4D.
- the transmission / reception of signals between the video distribution device 3 and the HMD_4A for user A to the HMD_4C for user C and the notebook PC_4D for user D is not performed by direct wireless communication, but via a public network. Is done.
- each of the user A HMD_4A to the user C HMD_4C outputs a display range setting signal generated based on the detection result of the sensor that detects the movement of the head and the eyeball of the user A to C, from the ISP server A_7A to the ISP server. It is transmitted to the video distribution device 3 via each of C_7C and the Internet 6. Further, the user D notebook PC_4D transmits a display range setting signal generated by the user D by operating the input means of the user D notebook PC_4D to the video distribution device 3 via the ISP server D_7D and the Internet 6.
- the video distribution device 3 generates a 3D live based on the display range setting signal received from each of the HMD_4A for user A to the HMD_4C for user C and the all-sky 3D live video signal received from the aerial photography drone (1) _11.
- the video signal is transmitted to each of the user A HMD_4A to the user C HMD_4C via the Internet 6 and each of the ISP server A_7A to the ISP server C_7C.
- the video distribution device 3 generates a two-dimensional live video signal based on the display range setting signal received from the user D notebook PC_4D and the all-sky 3D live video signal received from the aerial photography drone (2) _12. , It is transmitted to the notebook PC_4D for user D via the Internet 6 and the ISP server D_7D.
- each of the HMD_4A for the user A to the HMD_4C for the user C and the notebook PC_4D for the user D generate a voice signal based on the result of sound collection by the integrally configured microphone, and the ISP server A_7A to the ISP server. It is transmitted to the video distribution device 3 via each of D_7D and the Internet 6.
- the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D via the Internet 6 and the ISP server A_7A to ISP server D_7D. Will be sent.
- FIG. 14 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the 14th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8, local system 9) It is the same as the thirteenth embodiment except for the point described.
- the transmission of the spherical 3D live video signal from the aerial drone (1) _11 and the aerial drone (2) _12 to the video distribution device 3 is performed by the drone control controller (1) _221 and the drone control. Direct wireless transmission, not via controller (2) _222.
- FIG. 15 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the fifteenth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8, local system 9) This is the same as that of the 14th embodiment except for the points described up to.
- signals are transmitted and received between the aerial photography drone (1) _11 and the drone control controller (1) _221, and between the aerial photography drone (2) _12 and the drone control controller (2) _222. Is not performed by direct wireless communication, but is performed via the video distribution device 3.
- the video distribution device 3 receives the spherical 3D live video signals received from the aerial drone (1) _11 and the aerial drone (2) _12, respectively, to the drone control controller (1) _221 and the drone control controller ( 2) Send wirelessly to _222.
- the drone control controller (1) _221 generates a flight control signal based on the lever operation and dial operation of the operator (1), and wirelessly transmits the flight control signal to the video distribution device 3.
- the video distribution device 3 wirelessly transmits the flight control signal received from the drone control controller (1) _221 to the aerial photography drone (1) _11 after performing necessary processing.
- the rotation speed of the rotor blades of the aerial drone (1) _11 is controlled based on the received flight control signal, and the aerial drone (1) _11 flies as intended by the operator (1).
- the drone control controller (2) _222 generates a flight control signal based on the lever operation and dial operation of the operator (2), and wirelessly transmits the flight control signal to the video distribution device 3.
- the video distribution device 3 wirelessly transmits the flight control signal received from the drone control controller (2) _222 to the aerial photography drone (2) _12 after performing necessary processing.
- the rotation speed of the rotor blades of the aerial drone (2) _12 is controlled based on the received flight control signal, and the aerial drone (2) _12 flies as intended by the operator (2).
- FIG. 16 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the 16th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, smartphone for user A_41A to smartphone for user D_41D, video archive device 8.
- the function of the local system 9) is the same as that of the thirteenth embodiment except for the points described up to paragraph [0176].
- the user A device to the user C device are composed of the user A smartphone_41A to the user C smartphone_41C in addition to the user A HMD_4A to the user C HMD_4C, and the user D device.
- Is composed of a smartphone for user D_41D in addition to a notebook PC_4D for user D, and HMD_4A for user A to HMD_4C for user C and notebook PC_4D for user D are smartphones for user A_41A to user D, respectively. It is connected to the smartphone _41D by wire with a communication cable.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D does not send and receive signals to and from each of the ISP server A_7A to the ISP server C_7C and the ISP server D_7D, respectively.
- Signals are transmitted and received to and from each of the 5G base station A_5A to 5G base station C_5C and the 5G base station D_5D via the smartphone _41A to the smartphone _41D for the user D.
- each of the HMD_4A for the user A to the HMD_4C for the user C outputs a display range setting signal generated based on the detection result of the sensor that detects the movement of the head and the eyeball of the users A to C to the smartphone_41A for the user A.
- the user D notebook PC_4D uses the display range setting signal generated by the user D by operating the input means of the user D notebook PC_4D to the user D smartphone_41D and the 5G base station D_5D, and the 5G network and the Internet. It is transmitted to the video distribution device 3 via 6.
- the video distribution device 3 generates a 3D live based on the display range setting signal received from each of the HMD_4A for user A to the HMD_4C for user C and the all-sky 3D live video signal received from the aerial photography drone (1) _11.
- the video signal is transmitted from the Internet 6 and the 5G network, each of the 5G base stations A_5A to 5G base station C_5C, and each of the user A smartphone_41A to the user C smartphone_41C, and the user A HMD_4A to the user C. Send to each of the HMD_4C for.
- the video distribution device 3 generates a two-dimensional live video signal based on the display range setting signal received from the user D notebook PC_4D and the all-sky 3D live video signal received from the aerial photography drone (2) _12. , Internet 6 and 5G network, and 5G base station D_5D and user D smartphone_41D to transmit to user D notebook PC_4D.
- each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D uses a smartphone_41A for the user A to generate an audio signal based on the result of sound collection by the integrally configured microphone.
- the superimposed audio signals A to D generated by the video distribution device 3 are the Internet 6 and the 5G network, each of the 5G base stations A_5A to 5G base stations D_5D, and the smartphones for user A_41A to the smartphones for user D_41D. It is transmitted to each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D via each of the above.
- FIG. 17 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the 17th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, smartphone for user A_41A to smartphone for user D_41D, video archive device 8.
- the function of the local system 9) is the same as that of the 14th embodiment except for the points described up to paragraph [0180].
- the user A device to the user C device are composed of the user A smartphone_41A to the user C smartphone_41C in addition to the user A HMD_4A to the user C HMD_4C, and the user D device.
- Is composed of a smartphone for user D_41D in addition to a notebook PC_4D for user D, and HMD_4A for user A to HMD_4C for user C and notebook PC_4D for user D are smartphones for user A_41A to user D, respectively. It is connected to the smartphone _41D by wire with a communication cable.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D does not send and receive signals to and from each of the ISP server A_7A to the ISP server C_7C and the ISP server D_7D, respectively.
- Signals are transmitted and received to and from each of the 5G base stations A_5A to 5G base stations C_5C and the 5G base stations D_5D via the smartphone A_41A to the smartphone _41D for the user D.
- FIG. 18 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the eighteenth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, smartphone for user A_41A to smartphone for user D_41D, video archive device 8.
- the function of the local system 9) is the same as that of the fifteenth embodiment except for the points described up to paragraph [0184].
- the user A device to the user C device are composed of the user A smartphone_41A to the user C smartphone_41C in addition to the user A HMD_4A to the user C HMD_4C, and the user D device.
- Is composed of a smartphone for user D_41D in addition to a notebook PC_4D for user D, and HMD_4A for user A to HMD_4C for user C and notebook PC_4D for user D are smartphones for user A_41A to user D, respectively. It is connected to the smartphone _41D by wire with a communication cable.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D does not send and receive signals to and from each of the ISP server A_7A to the ISP server C_7C and the ISP server D_7D, respectively.
- Signals are transmitted and received to and from each of the 5G base stations A_5A to 5G base stations C_5C and the 5G base stations D_5D via the smartphone A_41A to the smartphone _41D for the user D.
- FIG. 19 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the nineteenth embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8) will be described up to paragraph [0189]. It is the same as Example 15 except for the point.
- signals from the aerial drone (1) _11 and the aerial drone (2) _12 to the video distribution device 3 are transmitted and received not by direct wireless communication but via a public network. ..
- the aerial photography drone (1) _11 and the aerial photography drone (2) _12 transmit the omnidirectional 3D live video signal generated based on the results captured by the two omnidirectional cameras at the base station of the 5G network. It is transmitted to a certain 5G base station 5, and the spherical 3D live video signal is transmitted to the video distribution device 3 via the 5G network (not shown in the figure) and the Internet 6.
- the video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller (1) _221, and finally passes through the Internet 6 and the 5G network, and finally the 5G base station 5 Send to aerial drone (1) _11.
- the rotation speed of the rotor blades of the aerial drone (1) _11 is controlled based on the received flight control signal, and the aerial drone 1 flies as intended by the operator.
- the video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller (2) _222, and finally passes through the Internet 6 and the 5G network, and finally the 5G base station 5 Send to aerial drone (2) _12.
- the rotation speed of the rotor blades of the aerial drone (2) _12 is controlled based on the received flight control signal, and the aerial drone (2) _12 flies as intended by the operator.
- FIG. 20 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the 20th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D) Is the same as.
- the aerial drone (1) _11 and the aerial drone (2) _12 do not send and receive signals to and from only one 5G base station 5, but the 5G base stations I_5I to 5G base stations. Send and receive signals to and from III_5III in a _cellular manner.
- FIG. 21 describes the configuration and function of a live video distribution system that realizes the live video distribution method according to the twenty-first embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8) will be described up to paragraph [0199]. It is the same as Example 20 except for the point.
- the transmission and reception of signals between the operator (1) device 21 and the operator (2) device 22 and the video distribution device 3 are not performed by direct wireless communication, but through a public network. It is done through.
- the HMD_211 for the operator (1) transmits a display range setting signal generated based on the detection result of the sensor that detects the movement of the head and eyeballs of the operator (1) to the 5G base station 50, and displays the display.
- the range setting signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.
- the HMD_212 for the driver (2) transmits a display range setting signal generated based on the detection result of the sensor that detects the movement of the head and the eyeball of the driver (2) to the 5G base station 50, and displays the display.
- the range setting signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.
- the video distribution device 3 transmits the display range setting signal received from the HMD_211 for the driver (1) and the all-sky 3D live video signal received from the aerial photography drone (1) _11 to the HMD_211 for the driver (1).
- a 3D live video signal is generated, and the 3D live video signal is finally transmitted from the 5G base station 50 to the HMD_211 for the operator (1) via the Internet 6 and the 5G network.
- the video distribution device 3 transmits the display range setting signal received from the HMD_212 for the operator (2) and the all-sky 3D live video signal received from the aerial photography drone (2) _12 to the HMD_212 for the operator (2).
- a 3D live video signal to be transmitted is generated, and the 3D live video signal is finally transmitted from the 5G base station 50 to the HMD_212 for the operator (2) via the Internet 6 and the 5G network.
- the drone control controller (1) _221 transmits a flight control signal generated based on the operator's lever operation and dial operation to the 5G base station 50, and the flight control signal is transmitted via the 5G network and the Internet 6. It is transmitted to the video distribution device 3.
- the drone control controller (2) _222 transmits a flight control signal generated based on the operator's lever operation and dial operation to the 5G base station 50, and the flight control signal is transmitted via the 5G network and the Internet 6. It is transmitted to the video distribution device 3.
- the HMD_211 for the driver (1) and the HMD_212 for the driver (2) transmit a voice signal generated based on the result of sound collection by the integrally configured microphone to the 5G base station 50, and the above-mentioned
- the audio signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.
- the superimposed audio signal 0 generated by the video distribution device 3 is finally transmitted from the 5G base station 50 to the HMD_211 for the operator (1) and the HMD_212 for the operator (2) via the Internet 6 and the 5G network. Will be done.
- FIG. 22 describes the configuration and function of the live video distribution system that realizes the live video distribution method according to the 22nd embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- Components of the live video distribution system of this embodiment (aerial drone (1) _11, aerial drone (2) _12, HMD_211 for operator (1), HMD_212 for operator (2), controller for drone operation (1) ) _221, drone control controller (2) _222, video distribution device 3, HMD_4A for user A to HMD_4C for user C, notebook PC_4D for user D, video archive device 8) will be described up to paragraph [0203]. It is the same as Example 21 except for the point.
- the user A device to the user C device are composed of the user A smartphone_41A to the user C smartphone_41C in addition to the user A HMD_4A to the user C HMD_4C, and the user D device.
- Is composed of a smartphone for user D_41D in addition to a notebook PC_4D for user D, and HMD_4A for user A to HMD_4C for user C and notebook PC_4D for user D are smartphones for user A_41A to user D, respectively. It is connected to the smartphone _41D by wire with a communication cable.
- the smartphone for user A_41A to the smartphone for user D_41D have a 5G communication function.
- Each of the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D does not send and receive signals to and from each of the ISP server A_7A to the ISP server C_7C and the ISP server D_7D, respectively.
- Signals are transmitted and received to and from each of the 5G base stations A_5A to 5G base stations C_5C and the 5G base stations D_5D via the smartphone A_41A to the smartphone _41D for the user D.
- FIG. 23 describes the configuration and function of the live video distribution system that realizes the live video distribution method according to the 23rd embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for.
- the live video distribution method of the present invention is applied to the video distribution of the live video signal of an exhibit (including an art object and an exhibition article) exhibited in an exhibition hall (including a museum and a museum). It is a thing.
- the aerial photography drone is only the aerial photography drone 1, and the operator device is also only one set of the operator HMD_21 and the drone operation controller 22.
- the functions of HMD_4A for user A to HMD_4C for user C and notebook PC_4D for user D cannot be selected from a plurality of aerial photography drones for each of users A to C and user D, and the spherical image of the aerial photography drone 1 can be selected. This is the same as in the eighteenth embodiment, except that only the image of the live image signal of the object captured by the camera can be enjoyed.
- the exhibition hall is divided into the following three areas.
- the operator is located in the backyard, and the operator operates the drone control controller 22 while watching the video of the live video signal projected from the left and right projection units of the driver's HMD_21.
- the operator device (HMD_21 for operator and controller 22 for drone operation) and the video distribution device 3 transmit and receive signals in the same manner as in the second embodiment.
- the drone control antenna 23 is installed in the moving area.
- the drone control antenna 23 is connected to the video distribution device 3 by wire.
- the drone maneuvering antenna 23 mediates the transmission and reception of signals between the aerial photography drone 1 and the video distribution device 3.
- the aerial photography drone 1 and the video distribution device 3 can transmit and receive signals via the drone control antenna 23 in the same manner as in the second embodiment.
- the aerial photography drone 1 is equipped with an approach determination means for determining the approach to the exhibition area based on the results taken by the two spherical cameras. As a result, contact with the exhibits can be avoided.
- FIG. 24 shows the configurations and functions of a live video distribution system and a local system that realize the live video distribution method according to the 24th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment are the same as those of the 16th embodiment except for the points described up to paragraph [0225].
- the live video distribution system of this embodiment includes the local system ⁇ _9 ⁇ to the local system ⁇ _9 ⁇ as components, and these local systems are deployed in areas remote from each other.
- the local system ⁇ _9 ⁇ to the local system ⁇ _9 ⁇ are each equipped with a video distribution device ⁇ _3 ⁇ to a video distribution device ⁇ _3 ⁇ , and a plurality of operators who operate the aerial photography drone using the operator device are deployed.
- signals are exchanged between the video distribution device ⁇ _3 ⁇ to the video distribution device ⁇ _3 ⁇ and the user A HMD_4A to the user C HMD_4C and the user D notebook PC_4D via the Internet 6 and the video archive device 8.
- FIGS. 25 and 26 taking as an example user D using the user D notebook PC_4D and the user D smartphone_41D.
- User D operates the input means of the user D notebook PC_4D and inputs the ID and password.
- the input information is transmitted to the video archive device 8 via the user D smartphone_41D and the 5G base station D_5D, and the 5G network and the Internet 6, and as a result of being authenticated, the user D notebook PC_4D You can access the video archive device 8.
- the live video selection screen shown in FIG. 21 is displayed on the display of the notebook PC_4D for user D who has accessed the video archive device 8.
- the input information is the smartphone_41D for the user D, the 5G base station D_5D, and 5G. It is transmitted to the video archive device 8 via the network and the Internet 6.
- the video archive device 8 creates a list of applicable live videos based on the flight plans of all aerial drones belonging to the local system ⁇ _9 ⁇ to the local system ⁇ _9 ⁇ , and creates a list of applicable live videos, Internet 6 and 5G network, and 5G base station D_5D.
- a list of corresponding live images is displayed on the display of the user D notebook PC_4D that has received the list.
- the screen transitions to the video display screen.
- the screen transitions to the standby screen, and when the streaming distribution of the selected live video is started, the screen transitions to the video display screen.
- the aerial drone ⁇ 1_1 ⁇ 1 will quickly ascend by rotating each rotor based on the flight control signal transmitted from the drone control controller operated by the operator ⁇ 1. While moving over Shiretoko, it will start transmitting the "all-sky 3D live video signal of the drift ice to be shot” and the "live voice signal of the surrounding sound" to the video distribution device ⁇ _3 ⁇ .
- the aerial photography drone ⁇ 1_1 ⁇ 1 continuously generates a shooting position signal based on the measurement result of the GPS sensor provided by the aerial photography drone ⁇ 1_1 ⁇ 1 while transmitting the spherical 3D live image signal, and the spherical 3D live image. It is wirelessly transmitted to the video distribution device ⁇ _3 ⁇ together with the signal.
- the video distribution device ⁇ _3 ⁇ converts the received all-sky 3D live video signal into a two-dimensional live video signal, and then transmits the received live audio signal to the video archive device 8 via the Internet 6.
- the received spherical 3D live video signal and live audio signal are written as temporary data in the storage means provided by the player.
- the reception time of the spherical 3D live video signal and the live audio signal is set as the shooting time, and the shooting position is specified based on the shooting position signal transmitted from the aerial shooting drone ⁇ 1_1 ⁇ 1, and then the shooting time and shooting are performed. Write as video / audio data with position.
- the video distribution device ⁇ _3 ⁇ creates a video / audio data file from the temporary data at the time when the shooting of the aerial photography drone ⁇ 1_1 ⁇ 1 is completed and the signal transmission is completed, and the video archive device 8 is added to the video / audio data file. Send to.
- the video archive device 8 saves the transmitted video / audio data file.
- the video archive device 8 holds a list of video / audio data files stored by itself, and updates the list of the video / audio data files each time it receives the data files.
- the user D notebook PC_4D receives a two-dimensional live video signal and a live audio signal from the video archive device 8 via the Internet 6 and the 5G network, and the 5G base station D_5D and the user D smartphone_41D.
- the display screen is switched to the video display screen.
- the user D can display the live video in the display range he / she wants to see by operating the input means of the notebook PC_4D for the user D.
- the voice based on the received live voice signal is emitted from the microspeaker of the notebook PC_4D for user D.
- the video taken by the aerial drone ⁇ 1_1 ⁇ 1 (“Shiretoko drift ice tour (2)”) can be live-streamed to user devices used by users other than user D.
- the video archive device 8 counts the number of user devices that are currently livestreaming the video and the total number of user devices that are delivered during the livestreaming delivery time. Of these, the number of user devices currently being live-streamed is displayed in the "number of viewers" in FIG. 22, and the total number of user devices delivered during the live-streaming distribution time is a video. It is added as data to the video / audio data file saved in the archive device 8.
- the functions of the aerial drone constituting the local system ⁇ _9 ⁇ to the local system ⁇ _9 ⁇ , the HMD for the operator, the controller for operating the drone, and the video distribution device ⁇ _3 ⁇ to the video distribution device ⁇ _3 ⁇ constitute the 16th embodiment.
- the function is the same as that of the video distribution device 3, it can be the same as that of the 17th and 18th embodiments.
- the video archive device 8 has a function of transmitting the live video signal and the live audio signal to the user device, but causes the live video signal, the live audio signal, and the user device to transmit the live video signal and the live audio signal. It is also possible to transfer the function to a separately provided video distribution device.
- FIG. 27 shows the configurations and functions of the live video distribution system and the local system that realize the live video distribution method according to the 25th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method. It is a block diagram for demonstrating.
- Components of the live video distribution system of this embodiment are the same as in Example 16 except for the points described up to paragraph [0237]. It is the same.
- three aerial photography drones (1) _11 to aerial photography drones (3) _13 are deployed in the local system 9.
- Each of the aerial photography drone (1) _11 to the aerial photography drone (3) _13 has one CCD camera module, and the CCD camera module is installed on a 3-way pan head.
- the operators (1) to (3) who operate each of the aerial photography drone (1) _11 to the aerial photography drone (3) _13 are the drone operation controller (1) _221 to the drone operation controller (3) _223 (Fig.)
- the 3-way head can be rotated around the three rotation axes by operating (not described in). Accordingly, the tilt and pan of the front / rear / left / right of the CCD camera module can be realized, and an image of a desired imaging range can be captured.
- each aerial photography drone photographs the same building to be photographed from different viewpoints.
- the viewpoints (indicated by ⁇ in the figure) and the line of sight of each imaging means of the aerial drone (1) _11 to the aerial drone (3) _13 are shown in FIG. 28.
- Each of the aerial photography drone (1) _11 to the aerial photography drone (3) _13 generates a two-dimensional video signal based on the result imaged by the CCD camera module, and the two-dimensional video signal is used as the drone control controller (1). Radio transmission to the video distribution device 3 via _221 to drone control controller (3) _223.
- the video distribution device 3 receives the two-dimensional video signals received from each of the aerial photography drone (1) _11 to the aerial photography drone (3) _13 via the Internet 6 and the 5G network, and the smartphone for user A_41A. -Send to user D smartphone_41D at the same time.
- the user D can display only one live image of the live image 1 to the live image 3 in full screen on the display of the notebook PC_4D for the user D.
- the building is photographed from different viewpoints using a plurality of aerial photography drones, but for example, a concert or a sporting event held at one venue is different using a plurality of aerial photography drones. It can also be configured to shoot from the viewpoint.
- different live video signals are simultaneously received from a plurality of local systems deployed in regions remote from each other, and the corresponding live video is displayed in parallel, or one live video is selected from them. It can also be configured to display in full screen with.
- the present invention relates to an industry that manufactures unmanned mobile objects such as unmanned aerial vehicles (drones for aerial photography, etc.), unmanned vehicles, walking robots, peripheral devices and ancillary devices, an industry that manufactures video distribution devices, and devices for users (particularly HMD). It can be used in the industry that manufactures video display means such as) and the industry that manufactures video archiving equipment. It can also be used in industries that provide services related to observation using the live video distribution method (particularly tourism services).
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Studio Devices (AREA)
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2021553705A JPWO2021085572A1 (cg-RX-API-DMAC7.html) | 2019-10-30 | 2020-10-29 | |
| US17/772,046 US20220368958A1 (en) | 2019-10-30 | 2020-10-29 | Live video distribution method using unmanned moving device, video distribution device used in live video distribution method, and video archive device for storing video data file generated by video distribution device |
| JP2025012518A JP2025081342A (ja) | 2019-10-30 | 2025-01-29 | 無人移動体を用いたライブ映像配信方法、該ライブ映像配信方法において用いられる映像配信装置、及び、該映像配信装置が生成する映像データファイルを保存する映像アーカイブ装置 |
| JP2025080152A JP2025109846A (ja) | 2019-10-30 | 2025-05-13 | 無人移動体を用いたライブ映像配信方法、該ライブ映像配信方法において用いられる映像配信装置、及び、該映像配信装置が生成する映像データファイルを保存する映像アーカイブ装置 |
Applications Claiming Priority (10)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019197655 | 2019-10-30 | ||
| JP2019-197655 | 2019-10-30 | ||
| JP2019211894 | 2019-11-24 | ||
| JP2019-211894 | 2019-11-24 | ||
| JP2020090984 | 2020-05-26 | ||
| JP2020-090984 | 2020-05-26 | ||
| JP2020-095150 | 2020-05-31 | ||
| JP2020095150 | 2020-05-31 | ||
| JP2020-107590 | 2020-06-23 | ||
| JP2020107590 | 2020-06-23 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021085572A1 true WO2021085572A1 (ja) | 2021-05-06 |
Family
ID=75715211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2020/040720 Ceased WO2021085572A1 (ja) | 2019-10-30 | 2020-10-29 | 無人移動体を用いたライブ映像配信方法、該ライブ映像配信方法において用いられる映像配信装置、及び、該映像配信装置が生成する映像データファイルを保存する映像アーカイブ装置 |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US20220368958A1 (cg-RX-API-DMAC7.html) |
| JP (3) | JPWO2021085572A1 (cg-RX-API-DMAC7.html) |
| WO (1) | WO2021085572A1 (cg-RX-API-DMAC7.html) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2023004807A (ja) * | 2021-06-24 | 2023-01-17 | 仁宝電脳工業股▲ふん▼有限公司 | ドローンゲームのレンダリング方法 |
| JP2023095696A (ja) * | 2021-12-24 | 2023-07-06 | 株式会社エクシング | コンテンツ配信システム、及び、カラオケ再生システム |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220197893A1 (en) * | 2020-12-23 | 2022-06-23 | Here Global B.V. | Aerial vehicle and edge device collaboration for visual positioning image database management and updating |
| US11907888B2 (en) | 2020-12-23 | 2024-02-20 | Here Global B.V. | Aerial vehicle and computing device interaction for validating aerial vehicle activity |
| US20240406564A1 (en) * | 2023-06-02 | 2024-12-05 | SkyScope Corporation | Methods and systems for creating panoramic images for digitally exploring locations |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003267295A (ja) * | 2002-03-14 | 2003-09-25 | Foundation For Nara Institute Of Science & Technology | 遠隔操縦システム |
| JP6172783B2 (ja) * | 2014-07-31 | 2017-08-02 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | 無人航空機を用いて仮想観光をするシステムおよび方法 |
| JP2019104290A (ja) * | 2017-12-08 | 2019-06-27 | 浩幸 上野 | 無人航空機の仮想操縦システム |
| JP2019134383A (ja) * | 2018-02-02 | 2019-08-08 | トヨタ自動車株式会社 | 遠隔操作システム、及びその通信方法 |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2650493C2 (ru) * | 2013-06-07 | 2018-04-16 | Дап Риэлайз Инк. | Система распределения видео в реальном времени |
| JP6469962B2 (ja) * | 2014-04-21 | 2019-02-13 | 薫 渡部 | 監視システム及び監視方法 |
| US10554829B2 (en) * | 2014-06-03 | 2020-02-04 | Sony Corporation | Information processing device, photographing device, image sharing system, and method of information processing |
| WO2016026128A1 (en) * | 2014-08-21 | 2016-02-25 | SZ DJI Technology Co., Ltd. | Unmanned aerial vehicle communications methods and systems |
| US11263461B2 (en) * | 2015-10-05 | 2022-03-01 | Pillar Vision, Inc. | Systems and methods for monitoring objects at sporting events |
| JP6100868B1 (ja) * | 2015-11-09 | 2017-03-22 | 株式会社プロドローン | 無人移動体の操縦方法および無人移動体監視装置 |
| CN105828043A (zh) * | 2016-05-03 | 2016-08-03 | 铱格斯曼航空科技集团有限公司 | 一种无人机视频远程传输系统 |
| US10139836B2 (en) * | 2016-09-27 | 2018-11-27 | International Business Machines Corporation | Autonomous aerial point of attraction highlighting for tour guides |
| US20180152736A1 (en) * | 2016-11-30 | 2018-05-31 | Harold Glen Alexander | Live video recording, streaming, viewing, and storing mobile application, and systems and methods of use thereof |
| JP6754893B2 (ja) * | 2017-03-31 | 2020-09-16 | 本田技研工業株式会社 | 画像生成装置及び画像生成方法 |
| JP6677684B2 (ja) * | 2017-08-01 | 2020-04-08 | 株式会社リアルグローブ | 映像配信システム |
| US11403057B2 (en) * | 2017-10-31 | 2022-08-02 | Sony Corporation | Information processing device, information processing method, and information processing program |
| JP7059576B2 (ja) * | 2017-11-13 | 2022-04-26 | 大日本印刷株式会社 | 自動撮影システム及び自動撮影方法 |
-
2020
- 2020-10-29 JP JP2021553705A patent/JPWO2021085572A1/ja active Pending
- 2020-10-29 WO PCT/JP2020/040720 patent/WO2021085572A1/ja not_active Ceased
- 2020-10-29 US US17/772,046 patent/US20220368958A1/en not_active Abandoned
-
2025
- 2025-01-29 JP JP2025012518A patent/JP2025081342A/ja active Pending
- 2025-05-13 JP JP2025080152A patent/JP2025109846A/ja active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003267295A (ja) * | 2002-03-14 | 2003-09-25 | Foundation For Nara Institute Of Science & Technology | 遠隔操縦システム |
| JP6172783B2 (ja) * | 2014-07-31 | 2017-08-02 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | 無人航空機を用いて仮想観光をするシステムおよび方法 |
| JP2019104290A (ja) * | 2017-12-08 | 2019-06-27 | 浩幸 上野 | 無人航空機の仮想操縦システム |
| JP2019134383A (ja) * | 2018-02-02 | 2019-08-08 | トヨタ自動車株式会社 | 遠隔操作システム、及びその通信方法 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2023004807A (ja) * | 2021-06-24 | 2023-01-17 | 仁宝電脳工業股▲ふん▼有限公司 | ドローンゲームのレンダリング方法 |
| US11623150B2 (en) | 2021-06-24 | 2023-04-11 | Compal Electronics, Inc | Rendering method for drone game |
| JP7269300B2 (ja) | 2021-06-24 | 2023-05-08 | 仁宝電脳工業股▲ふん▼有限公司 | ドローンゲームのレンダリング方法 |
| JP2023095696A (ja) * | 2021-12-24 | 2023-07-06 | 株式会社エクシング | コンテンツ配信システム、及び、カラオケ再生システム |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2021085572A1 (cg-RX-API-DMAC7.html) | 2021-05-06 |
| US20220368958A1 (en) | 2022-11-17 |
| JP2025109846A (ja) | 2025-07-25 |
| JP2025081342A (ja) | 2025-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220368958A1 (en) | Live video distribution method using unmanned moving device, video distribution device used in live video distribution method, and video archive device for storing video data file generated by video distribution device | |
| JP6569881B2 (ja) | 無人航空機用端末装置、該無人航空機用端末装置によって制御される無人航空機、及び、無人航空機用端末装置用通信ユニット | |
| KR101955368B1 (ko) | 증강현실을 이용한 가상 드론 경기장 제공 시스템 및 그 방법 | |
| JP2019126082A5 (cg-RX-API-DMAC7.html) | ||
| WO2020090697A1 (ja) | 遠隔ライブ映像娯楽施設及び該遠隔ライブ映像娯楽施設を利用するユーザに対する課金方法 | |
| US12452481B2 (en) | Video display system, information processing device, information processing method, and recording medium | |
| CN108600607A (zh) | 一种基于无人机的消防全景信息展示方法 | |
| CN108628439A (zh) | 信息处理设备、信息处理方法及程序 | |
| CN111556298A (zh) | 一种无人机远程游览服务系统及方法 | |
| CN108650494A (zh) | 基于语音控制的可即时获取高清照片的直播系统 | |
| JP2018067060A (ja) | 仮想3次元模型を対象とした立体複合現実感提供プログラム | |
| WO2017068928A1 (ja) | 情報処理装置及びその制御方法、並びにコンピュータ・プログラム | |
| Pattanayak et al. | Comparative Analysis of ENG, EFP and Drone camera and its Impact in Television Production | |
| JP2019135605A (ja) | 撮影映像表示装置及び撮影映像表示方法 | |
| JP2019036857A (ja) | ライブ映像娯楽施設 | |
| KR102485362B1 (ko) | 드론의 영상 위치 타임머신 다중 중계 시스템. | |
| JPWO2018179312A1 (ja) | 画像生成装置及び画像生成方法 | |
| JP2022138018A (ja) | 無人航空機制御システム | |
| JP2021153221A (ja) | ドローン撮影映像提供システム、ドローン撮影映像提供システムのプログラム、移動体撮影映像提供システム、移動体撮影映像提供システムのプログラム | |
| JP2002077899A (ja) | 映像データ配信システム | |
| JP2019213207A (ja) | 無人移動体用端末装置、該無人移動体用端末装置によって制御される無人移動体、及び、無人移動体用端末装置用通信ユニット | |
| WO2024180639A1 (ja) | 撮影システム、撮影方法、移動体制御装置及びプログラム | |
| WO2021215483A1 (ja) | 無人移動体を用いた映像処理システム、映像処理方法及び映像処理装置 | |
| CN114666493A (zh) | 一种ar观览服务系统和终端 | |
| JP2021190049A (ja) | 仮想トラベルシステム(無人航空機×仮想現実×拡張現実×複合現実) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20881727 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2021553705 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20881727 Country of ref document: EP Kind code of ref document: A1 |