JP7397482B2 - Video processing system, video processing method, and video processing device using unmanned moving objects - Google Patents

Description

The present invention relates to a video processing system, a video processing method, and a video processing device using unmanned moving bodies, and particularly relates to a video processing system, a video processing method, and a video processing device using unmanned moving bodies, and in particular, the present invention relates to an image processing system, a video processing method, and a video processing device using unmanned moving bodies. The present invention relates to a processing system, a video processing method, and a video processing device.

In recent years, with the spread of lithium-ion batteries and the miniaturization and cost reduction of electronic devices such as microelectromechanical systems (MEMS), gyroscopes, and acceleration sensors, unmanned vehicles that can be easily controlled remotely have become popular. (e.g., drones) are becoming commercially available. Races to compete in the skill of operating unmanned vehicles are held in various parts of Japan and abroad, and are attracting attention as a new motor sport and as a form of familiar entertainment.

In unmanned mobile vehicle races (for example, drone races), the pilot wears a head-mounted display and can remotely control the drone while viewing real-time images transmitted from a camera mounted on the front of the drone. You can watch the real-time video on a large display.
In addition, in unmanned vehicle races, the lap time of each unmanned vehicle is measured and the measured lap time is displayed on a large display to enhance the competitiveness (for example, the radio control described in Patent Document 1 Examples include a car lap time measuring device and a bicycle race time measuring system described in Patent Document 2).

JP2009-118878A Japanese Patent Application Publication No. 2002-109594

By the way, when operating the above-mentioned unmanned vehicle race, it is expected that the operator and viewers will be able to watch real-time video transmitted from the camera mounted on the unmanned vehicle, which will provide even higher entertainment value. A more immersive production style was expected.
To be more specific, augmented reality (AR) technology allows virtual visual information to be superimposed on real images, and by further combining this augmented reality with radio control (RC) technology. , the presentation of an unmanned mobile vehicle race that provides a new experience that combines the real world and the virtual world has been expected.
It was expected that this type of presentation would be realized at an even higher definition level with the advent of high-capacity, low-latency communication technology through the 5th generation mobile communication system (5G), which has begun trial operation.

In addition, there has also been a demand for application to business as a new video processing system that uses a plurality of unmanned moving bodies equipped with photographing devices, without being particularly limited to the production mode of unmanned moving body races.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a novel video processing system, a video processing method, and a video processing device that use a plurality of unmanned moving objects equipped with photographing devices. There is a particular thing.
Another object of the present invention is to provide a video processing system, a video processing method, and a video processing device using unmanned moving objects that can provide higher entertainment in operating an unmanned moving object race. It is in.
Another object of the present invention is to provide a video processing system, a video processing method, and a video processing device using an unmanned moving object that can provide a new experience that combines the real world and the virtual world. be.

According to the video processing system using unmanned moving bodies of the present invention, the above problem can be solved by a plurality of unmanned moving bodies equipped with photographing devices and moving while photographing external images, and by wireless communication with the unmanned mobile bodies. A video processing system comprising: a video processing device connected to the unmanned mobile object to process the video shot by the unmanned moving object, the video processing device acquiring video data indicating the video shot by the unmanned moving object. a video data acquisition unit; a screen display unit that displays a video indicated by the acquired video data on a display screen; and a mark that detects the presence of an identification mark to be detected in the video indicated by the acquired video data. a detection unit; and a mobile body identification unit that identifies which unmanned mobile body is the unmanned mobile body provided with the detected identification mark when the identification mark is detected. , when detecting a predetermined identification mark in the video shot by the first unmanned moving object and identifying that the unmanned moving object provided with the predetermined identification mark is the second unmanned moving object, The problem is solved by the screen display unit displaying the image taken by the first unmanned moving object and displaying information regarding the second unmanned moving object at a target position on the image.
With the above configuration, it is possible to realize a video processing system using an unmanned moving object that can provide higher entertainment in operating an unmanned moving object race.
Specifically, by using the identification mark provided on each unmanned moving object, the video processing device displays the image taken by the first unmanned moving object, and also displays information regarding the second unmanned moving object. It can be displayed (extended display) at the target position on the video. Therefore, when operating an unmanned mobile vehicle race, it is possible to provide the pilots and viewers with a new experience that combines the real world and the virtual world.

At this time, the unmanned mobile object is remotely controlled by an operation terminal and moves along a predetermined course, and the mark detection section detects a second identification mark to be detected in the video indicated by the acquired video data. When the second identification mark is detected in the image, the video processing device causes the unmanned mobile object to pass through a passage gate provided with the detected second identification mark. The mark detection unit further includes a gate passage determination unit that determines that the second identification mark provided at the predetermined passage gate exists in the video shot by the first unmanned moving object. When it is detected that the first unmanned moving object has passed through the predetermined passage gate, the screen display unit may display the image and display content based on a determination result that the first unmanned moving object has passed through the predetermined passage gate. .
With the above configuration, by determining whether or not the unmanned moving object has passed through the passage gate using the second identification mark, the position of the unmanned moving object can be detected with high accuracy, and the timing of passing the predetermined position can be accurately determined. It can be measured.
In addition, in order to further enhance the entertainment value when operating an unmanned vehicle race, content based on the determination result that the unmanned vehicle has passed through the passage gate is displayed on the display screen, creating a realistic production mode. becomes possible.

At this time, the unmanned moving body includes a first imaging device that is attached to a front position or a central position of the unmanned moving body, and that shoots a first-person video from a first-person viewpoint from the unmanned moving body; or a second photographing device that is attached to a rear position of the main body of the unmanned moving body and shoots a third-person video from a third-person perspective including an image of the unmanned moving body itself, and the screen display unit is equipped with: It is preferable that the first-person video and the third-person video taken by the unmanned moving object be switched and displayed in response to a selection by a user operation.
With the above configuration, the operator (viewer) can switch between first-person and third-person images during the unmanned mobile vehicle race, thereby providing higher entertainment and a more immersive production mode when operating the unmanned mobile vehicle race. be able to.

At this time, the unmanned moving object is remotely controlled by an operating terminal, moves on a predetermined course, and is installed at a predetermined position on the course, and a plurality of external devices are installed at a predetermined position on the course and take pictures of the unmanned moving object moving on the course. The image data acquisition unit further includes a photographing device, and the video data acquisition unit acquires video data representing a video photographed by the external photographing device, and the screen display unit receives a selection by a user operation and displays a plurality of the unmanned moving objects, It is preferable that the images taken by the plurality of external photographing devices be switched and displayed.
With the above configuration, during the unmanned mobile vehicle race, the driver can not only check the images taken by the unmanned mobile object, but also check the images taken by a predetermined external photographing device located on the course. Furthermore, the viewer can select the unmanned moving object he/she wishes to view from among the plurality of unmanned moving objects, and view the video shot by the selected unmanned moving object.

At this time, the unmanned moving object is remotely controlled by an operating terminal and moves along a predetermined course, and the video processing device detects when the identification mark is detected in the video under a predetermined condition. an inoperability determining unit that determines that the unmanned moving object provided with the identification mark is inoperable, and returns the unmanned moving object from the inoperable state when it is determined that the unmanned moving object is inoperable. Therefore, it is preferable to further include an inoperability response unit that takes a predetermined response to the unmanned moving body.
With the above configuration, if a trouble occurs in a predetermined unmanned vehicle during an unmanned vehicle race and the unmanned vehicle becomes unable to be remotely controlled, trouble support can be provided to the unmanned vehicle. . Specific trouble support includes notifying the operator of an unmanned mobile vehicle race that a problem has occurred, and sending a waiting drone to the location where the problem occurred to rescue the unmanned vehicle. , it is conceivable to control it so that it is collected.

At this time, the unmanned moving objects are remotely controlled by an operating terminal to move along a predetermined course, and the image processing device is configured to move a plurality of unmanned moving objects along a predetermined course under predetermined conditions. The mobile body control unit further includes a mobile body control unit that performs predetermined response control for the unmanned mobile bodies, and the mobile body control unit controls the plurality of unmanned mobile bodies in order to start moving the plurality of unmanned mobile bodies at a predetermined timing. It is preferable to control the movement of the moving body.
With the above configuration, the moving operations of a plurality of unmanned moving bodies can be controlled as necessary in order to smoothly operate the unmanned moving body race.
For example, at the start of an unmanned vehicle race, the movement of each unmanned vehicle is supported and controlled until each unmanned vehicle reaches a predetermined starting position, and the drive unit of each unmanned vehicle is controlled until the green signal lights up. It can be controlled so that it does not operate.

At this time, the unmanned moving object is remotely controlled by an operating terminal, moves along a predetermined course, and is provided on the front and rear surfaces of the unmanned moving object, respectively, and the object comes into contact with the unmanned moving object. The image processing device further includes a detection sensor for detecting that the plurality of unmanned moving objects is moved in a predetermined manner in order to move the plurality of unmanned moving objects in a course under predetermined conditions. The mobile body control unit further includes a body control unit, and when receiving a detection signal of the detection sensor from a predetermined unmanned mobile body, the mobile body control unit limits or stops the movement operation of the predetermined unmanned mobile body for a predetermined period of time. It's good to have control.
With the above configuration, when predetermined unmanned moving objects come into contact with each other during an unmanned moving object race, when a predetermined unmanned moving object contacts the course side, etc., a penalty can be imposed on the unmanned moving object. Specific penalties include limiting (decelerating) or stopping the movement of the unmanned vehicle until a predetermined time has elapsed (penalty stop), adding additional time to the lap time (time penalty), and the like.
It is also conceivable to change the content of the penalty depending on the degree of impact when the unmanned moving object makes contact.
By doing so, it is possible to provide a higher level of entertainment and a more immersive production mode when operating an unmanned mobile race.

Further, the above-mentioned problem is directed to a video processing method in which a computer connected by wireless communication to a plurality of unmanned mobile bodies equipped with a photographing device and moving while photographing external images processes the images photographed by the unmanned mobile bodies. a video data acquisition step in which the computer acquires video data showing the video shot by the unmanned mobile object; and a first screen display step in which the computer displays the video shown by the acquired video data on a display screen. a mark detection step of detecting the presence of an identification mark to be detected in a video indicated by the acquired video data; and when the identification mark is detected, the unmanned vehicle on which the detected identification mark is provided. a moving object identification step of identifying which unmanned moving object the moving object is, and in the mark detection step, detecting a predetermined identification mark in the video shot by the first unmanned moving object; In the moving object identification step, when the unmanned moving object provided with the predetermined identification mark is identified as a second unmanned moving object, an image taken by the first unmanned moving object is The problem can also be solved by a video processing method using an unmanned moving object, which further comprises performing a second screen display step of displaying information regarding the second unmanned moving object at a target position on the image. Ru.

The above-mentioned problem is also an image processing device that is equipped with a photographing device, is connected by wireless communication to a plurality of unmanned moving bodies that move while photographing external images, and processes the images photographed by the unmanned mobile bodies, a video data acquisition unit that acquires video data representing a video captured by the unmanned mobile object; a screen display unit that displays video represented by the acquired video data on a display screen; a mark detection unit that detects the presence of an identification mark to be detected in the detection unit; and a mark detection unit that detects the presence of an identification mark to be detected; a mobile object identification unit that detects a predetermined identification mark in the video shot by the first unmanned moving object, and identifies the unmanned moving object provided with the predetermined identification mark. When identifying that the object is the second unmanned moving object, the screen display section superimposes information regarding the second unmanned moving object at a target position on the video shot by the first unmanned moving object. The problem can also be solved by an image processing device using an unmanned moving object, which is characterized by displaying images.

According to the present invention, it is possible to realize a new video processing system, a new video processing method, and a new video processing device using a plurality of unmanned moving objects each equipped with a photographing device.
Furthermore, it becomes possible to provide higher entertainment value when operating an unmanned mobile race.
It will also be possible to provide a new experience that combines the real world and virtual world.

FIG. 1 is a configuration diagram of the entire video processing system according to the present embodiment. FIG. 2 is a configuration diagram of an unmanned mobile object, an operation terminal, and a monitor. FIG. 2 is a diagram showing the positions of a photographing device, an identification mark, and a detection sensor attached to an unmanned moving body. It is a figure which shows the passage gate with an identification mark. It is a figure which shows the modification 1 of the passage gate with an identification mark. It is a figure which shows the modification 2 of the passage gate with an identification mark. FIG. 1 is a hardware configuration diagram of a video processing device. FIG. 3 is a software configuration diagram of the video processing device. FIG. 3 is a diagram showing a display screen displayed by a screen display section. FIG. 3 is a diagram showing a display screen from a first person perspective (FPV). FIG. 3 is a diagram showing a display screen from a third-person perspective (TPV). FIG. 3 is a diagram showing a display screen from a bird's-eye view (BEV). FIG. 3 is a diagram showing a switching display screen on which the entire course map is displayed. 8 is a diagram showing a state in which a monitor screen is displayed on the display screen of FIG. 7. FIG. 10 is a diagram showing a state in which an advertisement is displayed on the display screen of FIG. 9. FIG. 8 is a diagram showing a state in which a ghost image is displayed on the display screen of FIG. 7. FIG. FIG. 2 is a processing flow diagram illustrating an example of the video processing method of the present embodiment.

Embodiments according to the present invention will be described below with reference to FIGS. 1 to 13.
This embodiment includes a plurality of unmanned moving objects equipped with photographing devices that move while photographing external images, and an image processing device that is connected to the unmanned mobile objects through wireless communication and processes the images shot by the photographing devices. A video processing system comprising: a video data acquisition unit that acquires video data representing video captured by an unmanned moving object; and a video data acquisition unit that displays video represented by the acquired video data on a display screen. a screen display unit; a mark detection unit that detects the presence of an identification mark to be detected in a video indicated by the acquired video data; The unmanned moving object has a moving object identification unit that identifies which unmanned moving object it is, and detects a predetermined identification mark in the video shot by the first unmanned moving object, and When the unmanned moving object provided with the identification mark is identified as the second unmanned moving object, the screen display section displays the image taken by the first unmanned moving object, and also displays the image taken by the second unmanned moving object. This invention relates to an invention whose main feature is to display information regarding a moving object at a target position on a video.

The overall configuration of the video processing system S of this embodiment is shown in FIGS. 1-4A.
The video processing system S is a system for operating an unmanned mobile body race, and is a system for managing an unmanned mobile body 1 equipped with a photographing device 1a and flying while photographing external images, and an unmanned mobile body 1 through wireless communication with the unmanned mobile body 1. An operation terminal 10 for remotely controlling the unmanned mobile object 1, a video processing device 20 for processing external video captured by the photographing device 1a and displaying it on a display screen, and a video processing device 20 connected to each other. A monitor 30 serving as a display screen for the operator, a display 40 and a mobile terminal 50 serving as a display screen for viewers, and an external photographing device 60 installed at intervals on the course of the unmanned mobile race. It mainly consists of a passage gate 70.
In addition, when a predetermined unmanned moving object 1 becomes unable to be remotely controlled during an unmanned moving object race, the rescue drone 80 is moved to a predetermined position on the course in order to control the unmanned moving object 1 to be rescued (recovered). I'm waiting.

As shown in FIGS. 1 to 3, the unmanned moving object 1 is a small radio-controlled car (RC car) that travels on a predetermined course while mainly taking external images of its front side. It performs data communication with the terminal 10 and the video processing device 20.
A plurality of unmanned moving objects 1 are prepared, and in this system, five unmanned moving objects 1 participate in an unmanned moving object race and travel on a predetermined course.
Although the number of unmanned vehicles that can run at the same time is limited depending on the radio band used, with the practical application of 5G, operators (viewers) from all over the world will be able to participate in the competition, and unmanned vehicles that can run at the same time will be able to operate at the same time. There are no particular restrictions on the number of bodies.

The unmanned mobile object 1 mainly includes a photographing device 1a, a transmitting/receiving antenna 1b, a moving unit 1c, a driving unit 1d, a processor 1e, a battery 1f, an identification mark 1g, and a detection sensor 1h. and are respectively attached to the main body of the unmanned vehicle 1.

The photographing device 1a (first photographing device 1aa) is a small photographic camera, and is attached to the front of the main body of the moving body, and photographs an external image of the front side and records the image. Then, video data representing the video is created.
To be more specific, the photographing device 1a is a small wide-angle camera, and includes a first photographing device 1aa, a second photographing device 1ab, a third photographing device 1ac, and a fourth photographing device attached to the front, rear, left, and right outer surfaces of the main body of the moving body. It has a photographing device 1ad.
Note that the photographing device 1a is not particularly limited to a wide-angle camera, and may be, for example, a 360-degree camera. When a 360-degree camera is employed, the unmanned moving object 1 does not necessarily have to include a plurality of photographing devices.

The first photographing device 1aa is attached to the upper surface of the front position of the unmanned moving body 1, and photographs a “first person video” from a first person view (FPV) from the unmanned moving body 1 (see FIG. 8A).
The second photographing device 1ab is attached to the upper surface of a wing-shaped rear wing provided at a rear position of the unmanned vehicle 1. That is, it is attached at a rear position than the main body of the moving body. Then, the second photographing device 1ab photographs a "third person video" from a third person view (TPV) including a video of the unmanned moving object 1 itself (see FIG. 8B).
The third photographing device 1ac and the fourth photographing device 1ad are attached to the outer surface of the unmanned moving body 1 at the left and right side positions, respectively.
The photographing device 1a can generate a "bird's eye view video (bird's eye view composite video)" from a bird's eye view (BEV) by combining the videos taken by the photographing devices 1aa, 1ab, 1ac, and 1ad. Yes (see Figure 8C).

The transmitting/receiving antenna 1b is attached to the upper end of the mobile body, and is used to receive operation data from the operation terminal 10 and to transmit captured video data to the video processing device 20 and monitor 30.
The moving unit 1c has four wheels attached to the lower ends of the four corners of the main body of the moving body, and receives driving power from the drive unit 1d to rotate and change direction left and right, thereby moving the unmanned moving body. 1 in the front, rear, left and right directions.
The drive unit 1d is a motor for driving the mobile unit 1c, is connected and attached to the mobile unit 1c, and operates based on a drive command received from the processor 1e.

The processor 1e is a microprocessor mainly composed of a CPU as a data arithmetic/control processing device, ROM, RAM, and HDD as storage devices, and a communication interface for transmitting and receiving information data through a transmitting/receiving antenna 1b. A processor installed inside the mobile body.
The battery 1f is a lithium ion battery for supplying power to the transmitting/receiving antenna 1b, the drive unit 1d, and the processor 1e, and is attached to the lower part of the mobile body.

The identification mark 1g is a two-dimensional barcode, which is attached to the top, bottom, front, back, left, and right outer surfaces of the main body of the moving body, and identifies which unmanned moving body 1 it is among the plurality of unmanned moving bodies 1, and also identifies the unmanned moving body 1. Identification data for identifying the position of the moving body 1 is stored.
The detection sensor 1h is a contact sensor (pressure sensor) that detects that an object has come into contact with the unmanned vehicle 1, and is attached to the front, rear, left, and right outer surfaces of the vehicle body, respectively.
Specifically, the detection sensors 1h are attached to the front surface of the front bumper, the rear surface of the rear bumper, and the outer surfaces of the left and right side bumpers of the unmanned moving object 1, respectively.
A detection signal detected by the detection sensor 1h is transmitted from the unmanned moving object 1 to the video processing device 20.

As shown in FIGS. 1 and 2, the operating terminal 10 is a controller operated by an operator, and is provided for each unmanned vehicle 1, and controls the unmanned vehicle 1 to travel on a predetermined course. It is operated remotely using wireless communication.
Specifically, the operation terminal 10 receives user operation input from the operator, creates operation data for operating the unmanned mobile object 1, and transmits the operation data to the unmanned mobile object 1. I can do it.
Further, the operation terminal 10 can perform data communication with the video processing device 20, accept user operation input, and switch the video displayed on the display screen of the monitor 30.

As shown in FIGS. 1 to 3, the video processing device 20 is a computer that performs data communication with the unmanned moving object 1, the operating terminal 10, the monitor 30, the display 40, the mobile terminal 50, the external photographing device 60, and the rescue drone 80. A video (enhanced video) in which predetermined "visual information" is superimposed on "real video" shot by the camera 1a and the external camera 60 is displayed on the monitor 30, display 40, and mobile terminal 50. be.
Specifically, as shown in FIG. 7, the video processing device 20 detects the identification mark 1g attached to the second unmanned moving object 1 in the real video shot by the first unmanned moving object 1. At times, individual information regarding the second unmanned moving object 1 is acquired. Then, the individual information regarding the second unmanned moving object 1 can be displayed superimposed on the "real video" taken by the first unmanned moving object 1.
Further, as shown in FIG. 7, when a second identification mark 70A, which will be described later, is detected in the real video shot by the first unmanned moving object 1, the video processing device 20 detects the second identification mark 70A. It is determined that the vehicle has passed through the passage gate 70 to which the vehicle is attached. Then, content based on the determination result that the first unmanned moving object 1 has passed through the passage gate 70 can be simultaneously displayed on the "real video".

The monitor 30 is a head-mounted display that is attached to the head of the operator, is provided for each unmanned moving object 1, performs data communication with the video processing device 20, and displays the video (extended) generated by the video processing device 20. images) can be displayed on the display screen.
Specifically, a display screen as shown in FIG. 7 can be displayed in real time.
Note that the monitor 30 is not particularly limited to a head-mounted display, and any known display device can be used. For example, it may be a cockpit-type display device integrated with the operating terminal 10.

The display 40 is a large display connected to the video processing device 20, and is used as a display screen for viewers viewing the unmanned mobile race.
Specifically, display screens such as those shown in FIGS. 7 to 9 are displayed in real time on the display 40, making it possible to produce content full of realism of the unmanned mobile body race.
The mobile terminal 50 is a tablet terminal connected to the video processing device 20, and, like the display 40, is used as a display screen for viewers.
Note that the mobile terminal 50 is used by each viewer, and for example, the user selects the unmanned moving object 1 he or she wishes to view from among the plurality of unmanned moving objects 1 (see FIG. 9), and Videos taken by the mobile object 1 can be viewed.

The external photographing devices 60 are large-sized photographing cameras for photographing each unmanned moving object 1 during the race, and a plurality of external photographing devices 60 are installed at predetermined intervals on the course.
The external photographing device 60 photographs a video (real video) in a predetermined section on the course and records the video. Then, video data representing the video is created and the data is transmitted to the video processing device 20.

As shown in FIG. 4A, the passage gates 70 are gates through which the unmanned moving object 1 passes, and a plurality of passage gates 70 are installed at predetermined intervals on the course.
The passage gate 70 has a bridge shape and includes a pair of gate legs 71 and a gate connection part 72 that connects the upper portions of the pair of gate legs 71, and connects the pair of gate legs 71 and the gate. The area surrounded by the portion 72 is a passing area.
A plurality of second identification marks 70A are arranged at predetermined intervals on the front side of the gate connecting portion 72 located on the start side in the direction of travel of the course.

The second identification mark 70A is a two-dimensional barcode, and stores identification data for identifying which one of the plurality of passage gates 70 installed on the course.
Therefore, when the video processing device 20 detects the second identification mark 70A in the video shot by the unmanned moving object 1, it is possible to specify which passage gate 70 the unmanned moving object 1 has passed through. .

Note that the passage gate 70 is not particularly limited to a bridge-shaped passage gate, and may be changed, for example, to a pole-shaped passage gate 170 as shown in FIG. 4B.
Alternatively, for example, as shown in FIG. 4C, a passage gate 270 with a louver may be used.
If the passage gate 270 with the louver 271 is adopted, when the unmanned mobile body 1 reaches a position close to the passage gate 270, it is possible for the unmanned mobile body 1 to photograph the second identification mark 70A of the passage gate 270. becomes. As a result, the video processing device 20 can detect the position information of the unmanned moving object 1 with higher accuracy.

<Hardware configuration of video processing device>
As shown in FIG. 5, the video processing device 20 includes a CPU as a data calculation/control processing device, ROM, RAM, and HDD (SSD) as storage devices, and transmits and receives information data through a home network or the Internet. The computer has a communication interface that performs communication.
The video processing device 20 also includes a display device that displays character or image information displayed in a predetermined format, an input device that is operated by the user when inputting a predetermined command to the CPU, and an external hard disk or the like. It further includes a storage medium device and a printing device that outputs text or image information, and is also connected to a monitor 30 as a display screen, a display 40, and a mobile terminal 50.

As shown in FIG. 6, the ROM, HDD, and external storage device of the video processing device 20 store a video processing program in addition to a main program that performs the functions necessary for a computer. By being executed, the functions of the video processing device 20 will be exhibited.

<Software configuration of video processing device>
As shown in FIG. 6, the video processing device 20 stores "video data", "individual data", "lap time data", "current position data", as well as various programs and various data. a storage section 21 for storing "video data", a video data acquisition section 22 for acquiring "video data" from the unmanned mobile object 1, and a screen display section 23 for processing the video indicated by the acquired "video data" and displaying it on a display screen. , a mark detection unit 24 that detects the presence of the identification mark 1g (second identification mark 70A) in the video indicated by the acquired "video data", and when the identification mark 1g is detected in the predetermined video, The main component includes a moving body identification unit 25 that identifies which unmanned moving body 1 is the unmanned moving body 1 provided with the detected identification mark 1g.
Furthermore, when the second identification mark 70A is detected in a predetermined video, the video processing device 20 assumes that the unmanned moving object 1 has passed through the passage gate 70 provided with the detected second identification mark 70A. A gate passage determination unit 26 that makes a determination, and a calculation unit 27 (elapsed time calculation unit 27a) that calculates the elapsed time required for the unmanned moving object 1 to pass from a predetermined position to a predetermined passage gate 70 from the determination result. , a current position calculation unit 27b that calculates the current position of the unmanned moving object 1 on the course from the determination result.
Further, the video processing device 20, as a trouble processing unit 28, when the identification mark 1g is detected in a predetermined video under predetermined conditions, the unmanned moving body 1 equipped with the detected identification mark 1g is remotely operated. The inoperability determining unit 28a determines that the unmanned vehicle 1 has become inoperable, and takes a predetermined response to the unmanned vehicle 1 in order to recover the unmanned vehicle 1 from the inoperable state when it is determined that the unmanned vehicle 1 is inoperable. It further includes an inoperability response unit 28b.
The video processing device 20 further includes a mobile object control unit 29 that performs predetermined response control for the plurality of unmanned mobile objects 1 in order to move the plurality of unmanned mobile objects 1 on a course under predetermined conditions. There is.
These are composed of a CPU, ROM, RAM, HDD, communication interface, various programs, and the like.

In addition, to explain the unmanned mobile object 1 from a functional perspective, it includes a storage section 2 that stores various programs and various data, an operation data reception section 3 that acquires "operation data" from the operation terminal 10, and a "video data" storage section 2 that stores various programs and various data. '' to the video processing device 20.

The “video data” stored in the storage unit 21 is video data showing “real images” shot by each unmanned moving body 1 and each external photographing device 60, and is video data showing “real images” taken by each unmanned moving body 1 and each external photographing device 60, and is and is transmitted in real time from each external photographing device 60, and is centrally managed and stored in the storage unit 21.
Note that the video data (video data) is set to have a number of frame images per second of 30 (30FPS (Frame Per Second)), for example.
By referring to the video data, as shown in FIG. 7, there is a function to display "real images" taken by each unmanned moving object 1, a function to determine whether each unmanned moving object 1 has passed through a gate, and a lap time/current position calculation. function, inoperability determination function, etc.

“Individual data” is data indicating individual information regarding each unmanned moving body 1 in the unmanned moving body race, is created for each unmanned moving body 1, and is centrally managed and stored in the storage unit 21.
To be more specific, the individual data includes, as individual information regarding each unmanned moving object 1, information on the operator of each unmanned moving object 1 (image of the operator, name of the operator (Player1-Player4)), as well as information on the driver during the race. Information such as the fastest lap time (FT: Fastest Time) of each unmanned moving object 1, the current ranking during the race, and the time difference (Behind) with the own unmanned moving object 1 is included.
By referring to the individual data, for example, as shown in FIG. You can use the function to display overlapping information.

“Lap time data” is data indicating the lap time of each unmanned moving body 1 during the unmanned moving body race, is created for each unmanned moving body 1 by the elapsed time calculation unit 27a, and is stored in the storage unit 21. Centrally managed and stored.
Specifically, the lap time data includes the elapsed time (section lap time) required for each unmanned moving object 1 to pass from a predetermined start position to pass a predetermined passing gate 70, and the time required for each unmanned moving object 1 to complete one lap around the course from the start position. Contains information on the elapsed time required (1st, 2nd, and 3rd lap times), and the elapsed time required from the start position to the goal position (total lap time required to complete 5 laps around the course). In addition, information such as the elapsed time (section lap time) required from passing through the passing gate 70 located on the start position side to passing through the next passing gate 70 among adjacent passing gates 70 is also included.
By referring to the lap time data, as shown in FIG. 7, a function is provided to display various lap times of each unmanned moving object 1, the fastest lap time (FT), the ranking of each unmanned moving object 1, etc. on the display screen. can be used.

"Current position data" is data indicating the current position of each unmanned moving object 1 on the course of each unmanned moving object race, and is created for each unmanned moving object 1 by the current position calculation unit 27b. , are centrally managed and stored in the storage unit 21.
To be more specific, the current position data includes position information indicating which passage gate 70 each unmanned moving object 1 is located on the course (which passage gate 70 it is located at).
By referring to the current position data, it is possible to utilize a function of displaying the current position of each unmanned moving body 1 (current position on the course map) on the display 40, as shown in FIG.

The video data acquisition unit 22 acquires "video data" from each unmanned moving object 1 and each external imaging device 60, and the acquired video data is transmitted for each unmanned moving object 1 and each external imaging device 60. The information is classified and stored in the storage unit 21.

The screen display section 23 includes, as specific functional sections, a video display section 23a, a virtual information display section 23b, an elapsed time display section 23c, and a current position display section 23d.
The video display unit 23a displays the video shown by the "video data" acquired from each unmanned moving object 1 and each external photographing device 60 on the monitor 30, display 40, and mobile terminal 50 in real time.
To be more specific, the video display section 23a can receive a selection by a user operation and switch and display the respective "real videos" photographed by each unmanned moving object 1 and each external photographing device 60.
In addition, the video display unit 23a receives a selection by a user operation, and displays a "first-person video" photographed by the first photographing device 1aa of a predetermined unmanned moving object 1, and a "third-person video photographed by the third photographing device 1ac." ” and an “overhead video” obtained by combining videos taken by a plurality of photographing devices 1aa-1ad.

When looking at the present embodiment shown in FIG. 7 as a display screen on the monitor 30, a "real video" captured by the first unmanned moving object 1 (the unmanned moving object of Player 1) is displayed.
In addition, in the lower right part of the display screen in FIG. 7, there is an "FPV" operation button 31 for switching to "first-person video" of the first unmanned moving object 1, and a "TPV" operation button for switching to "third-person video". A button 32 and a "BEV" operation button 33 for switching to "overhead video" are displayed.
By accepting a user's selection of the "FPV" operation button 31 on the display screen of FIG. 7, the display screen can be switched to the display screen shown in FIG. 8A. Similarly, by accepting a user's selection of the "TPV" operation button 32 and "BEV" operation button 33, it is possible to switch to the display screen shown in FIGS. 8B and 8C.

The virtual information display unit 23b displays the individual information of the first unmanned moving object 1 in a superimposed manner in the "real video" taken by the first unmanned moving object 1, and also displays the information of the second unmanned moving object 1 identified. When the second unmanned moving object 1 is displayed, the individual information of the second unmanned moving object 1 is displayed in an overlapping manner.
Looking at the display screen in FIG. 7, individual information 34 (operator's image, name, current ranking) of the first unmanned moving object 1 is displayed in an overlapping manner in the upper part of the display screen. Note that information on the lap average speed of the first unmanned moving object 1 is also displayed in the lower left portion of the display screen.
In addition, individual information 35 of the second unmanned moving object 1 (operator's name, fastest lap time) is displayed at the target position where the second unmanned moving object 1 (unmanned moving object of Player 2) is displayed on the display screen in FIG. (FT) and time difference (Behind)) are displayed in an overlapping manner.

The elapsed time display section 23c and the current position display section 23d display on the display screen "contents based on the determination result" in which each unmanned mobile object 1 is determined to have passed through a predetermined passage gate 70 by the gate passage determination section 26. indicate.
To be more specific, the elapsed time display section 23c displays "contents related to the elapsed time of each unmanned moving object 1" calculated by the elapsed time calculation section 27a on the display screen in real time as content based on the above determination result.
Further, the current position display unit 23d can display on the display screen in real time the “content regarding the current position of each unmanned mobile object 1” calculated by the current position calculation unit 27b as the content based on the above determination result. .
When looking at the display screen of FIG. 7, the entire race information 36 of each unmanned moving object 1 (current ranking of each unmanned moving object 1) is displayed in the upper left part of the display screen as "contents related to the elapsed time of each unmanned moving object 1". , driver information, and fastest lap time (FT) are displayed.
In addition, in the upper right part of the display screen in FIG. Lap time, current lap time) are displayed.
In addition, in the upper right part of the display screen in FIG. 7, the course map 38 of the unmanned moving object race and each unmanned moving object moving in real time on the course map 38 are displayed as "contents regarding the current position of each unmanned moving object 1." 1 position display icon 38a is displayed.
Note that on the course map 38, a position display icon 38b of the external photographing device 60 is also displayed as content regarding the installation position of the external photographing device 60.

Further, the screen display unit 23 can also display a switching display screen shown in FIG. 9 in which the entire course map is displayed on the display 40 and the mobile terminal 50 as a display screen for viewers.
On the right side of the display screen in FIG. 9, there are operation buttons 41 for switching to images of each unmanned moving object 1 (Player 1 to Player 4) and operation buttons for switching to images of each external photographing device 60 (Camera 1 to Camera 4). 42 is displayed.
Further, on the left side of the display screen in FIG. 9, overall race information 43 of each unmanned moving object 1 (current ranking of each unmanned moving object 1, driver information, fastest lap time (FT)) is displayed.
Further, in the center part of the display screen in FIG. 7, a course map 44 of the unmanned mobile body race, position display icons 44a of each unmanned mobile body 1 moving on the course map 44 in real time, and positions of each external photographing device 60 are displayed. A display icon 44b is displayed.

The mark detection unit 24 detects the presence of the identification mark 1g and the second identification mark 70A to be detected in the video indicated by the acquired "video data".
Specifically, when the video data (video data) is set to have a frame image count of 30 frames per second (30FPS), the mark detection unit 24 detects 1g of identification marks within the frame image for each frame image. And the presence of the second identification mark 70A is detected.

The moving body identification unit 25 identifies which unmanned moving body 1 on which the detected identification mark is provided is the unmanned moving body 1 when the identification mark 1g is detected.
The identification mark 1g stores identification data for specifying which unmanned moving object 1 it is. The identification marks 1g are attached to the top, bottom, front, back, left and right of each unmanned moving body 1, respectively.
Therefore, as shown in FIG. 7, when the mark detection unit 24 detects the identification mark 1g on the right side and rear side of the second unmanned moving object 1 in the video shot by the first unmanned moving object 1, In addition, the moving object identification unit 25 can identify that the second unmanned moving object 1 is running diagonally to the left in front. In other words, in addition to acquiring the individual information of the second unmanned moving body 1, relative position information of the second unmanned moving body 1 (relative distance to the first unmanned moving body 1) can be acquired.

When the second identification mark 70A is detected in the video indicated by the acquired "video data" and the second identification mark 70A is no longer detected in the video after the video, the gate passage determination unit 26 detects the second identification mark 70A. , it is determined that the predetermined unmanned moving object 1 has passed through the passage gate 70 provided with the detected second identification mark 70A.
The second identification mark 70A stores identification data for specifying which passage gate 70 it belongs to. The size of the second identification mark 70A is determined in advance.
Therefore, when the mark detection unit 24 detects the predetermined second identification mark 70A in the frame image photographed by the predetermined unmanned moving object 1, the unmanned moving object 1 is located at or around any passing gate 70. You can identify where you are. Then, the position of the unmanned moving body 1 can be identified with high accuracy based on the size of the detected second identification mark 70A.

The calculation unit 27 has an elapsed time calculation unit 27a and a current position calculation unit 27b as specific functional units.
The elapsed time calculation unit 27a calculates the elapsed time (lap time) required for the predetermined unmanned moving object 1 to pass from the predetermined start position to the predetermined passage gate 70 from the above determination result by the gate passage determination unit 26. .
Specifically, the elapsed time calculation unit 27a calculates the elapsed time (lap time) and creates "lap time data" indicating the elapsed time.
As mentioned above, the "lap time data" includes the elapsed time (section lap time) required for each unmanned moving object 1 to pass from a predetermined start position to pass a predetermined passage gate 70, and the time required for each unmanned moving object 1 to pass through a predetermined passage gate 70 from the start position, Information such as the elapsed time required to complete the course (1st, 2nd, and 3rd lap times), the elapsed time required from the start position to the goal position (total lap time required to complete 3 laps around the course), etc. include.

The current position calculation unit 27b calculates the current position of the unmanned mobile object 1 within a predetermined space from the above determination result by the gate passing determination unit 26.
More specifically, the current position calculation unit 27b calculates the current position and creates "current position data" indicating the current position.
As described above, the "current position data" includes a position indicating which passing gate 70 each unmanned moving object 1 is located on (the surrounding position of which passing gate 70) each unmanned moving object 1 is located on the course of the unmanned moving object race. Contains information.
Note that when the current position calculation unit 27b calculates the current position of a predetermined unmanned moving object 1, it uses the lap time of the unmanned moving object 1 during the race, the lap time of the driver operating the unmanned moving object 1 in past races, etc. By also referring to the information, the current position of the unmanned moving object 1 can be calculated with higher accuracy.
By calculating the current position of the unmanned vehicle 1 in this way, the current position display icon 38a on the course map 38 can be displayed while being moved with high accuracy on the display screen shown in FIG.

The trouble processing unit 28 includes an inoperability determining unit 28a and an inoperability handling unit 28b as specific functional units.
When the identification mark 1g is detected under predetermined conditions in the video indicated by the acquired "video data", the inoperability determining unit 28a determines that the unmanned mobile object 1 on which the detected identification mark 1g is provided is incapable of being remotely operated. It is determined that it has become.
Specifically, when the identification mark 1g is detected, for example, at a predetermined position and in a predetermined orientation in the video shot by each unmanned moving object 1 and each external imaging device 60, the inoperability determination unit 28a detects the identification mark 1g. It is determined that the unmanned moving object 1 provided with the identification mark 1g is no longer remotely controllable.
Note that the difference between the first video shot by the unmanned moving object 1 and the second video after the first video is detected, and if the difference is less than a predetermined threshold and the unmanned moving object 1 is It is also good to refer to the determination result when it is determined that the vehicle is in a stopped state. By doing so, the accuracy of the determination result by the inoperability determining section 28a will be further improved.

The inoperability response unit 28b performs a predetermined response to the unmanned moving body 1 in order to recover the unmanned moving body 1 from the remote-inoperable state when it is determined that the unmanned moving body 1 is not remotely operable.
Specifically, the inoperability response unit 28b notifies the operator and viewers of the unmanned mobile race that a trouble has occurred, and also directs the rescue drone 80 on standby to the trouble point. At the same time, the unmanned vehicle 1 is controlled to be rescued and recovered.
Note that the rescue drone 80 may be controlled to receive user operation input regarding the occurrence of a trouble from the operator of the unmanned mobile object 1 that has become incapable of remote control, and to rescue and recover the unmanned mobile object 1. .

The moving body control unit 29 performs predetermined response control for each unmanned moving body 1 in order to move all the unmanned moving bodies 1 participating in the unmanned moving body race on a course under predetermined conditions, As specific functional units, it has a support control unit 29a and a penalty control unit 29b.
The support control unit 29a controls the movement of the plurality of unmanned moving bodies 1 in order to cause all the unmanned moving bodies 1 to start moving at a predetermined timing at the start of a race.
To be more specific, the support control unit 29a supports and controls the moving operation of each unmanned moving body 1 until each unmanned moving body 1 reaches a predetermined starting position. Then, support control can be performed so that the drive unit 1d of each unmanned moving object 1 does not operate until the green signal lights up.
Note that support control may be performed so that the drive unit 1d of each unmanned moving body 1 does not operate even after each unmanned moving body 1 reaches the goal.

When the penalty control unit 29b receives a detection signal from the detection sensor 1h from a predetermined unmanned moving object 1, it performs penalty control to limit or stop the moving operation of the unmanned moving object 1 for a predetermined period of time.
To be more specific, the penalty control unit 29b controls the unmanned moving body 1 when predetermined unmanned moving bodies 1 come into contact with each other or when the predetermined unmanned moving bodies 1 contact the course side during the unmanned moving body race. Penalty control can be performed for Specifically, control is performed to limit (decelerate) or stop the movement of the unmanned mobile object 1 until a predetermined time has elapsed (penalty stop), and control to add additional time to the lap time (time penalty) is performed. .
At this time, the content of the penalty control performed on the unmanned moving body 1 may be changed depending on the positions of the detection sensors 1h attached to the front, rear, left, and right sides of the unmanned moving body 1.
For example, when the detection sensor 1h on the front of the unmanned moving object 1 receives a detection signal (when the unmanned moving object 1 causes a contact accident), the moving operation of the unmanned moving object 1 is controlled to impose a heavy penalty. That's good. Further, when the detection sensor 1h on the rear surface of the unmanned moving body 1 receives a detection signal (when a contact accident occurs), it is preferable to control the moving operation of the unmanned moving body 1 so as to impose a light penalty.
Note that the content of the penalty control performed on the unmanned moving object 1 may be changed depending on the degree of impact when the unmanned moving object 1 makes contact.

In addition, the mobile object control section 29 preferably further includes a voice communication control section 29c, an advertisement display control section 29d, and a ghost display control section 29e as specific functional sections.
When the voice communication control unit 29c detects the identification mark 1g of the second unmanned moving object 1 in the video shot by the first unmanned moving object 1, the voice communication control unit 29c communicates with the operator of the first unmanned moving object 1, Control is performed so that voice communication with the operator of the second unmanned mobile object 1 is possible for a predetermined period of time.
To be more specific, the monitor 30 worn by the pilot is provided with a microphone section (voice transmitting section) and a receiver section (voice receiving section) (not shown), so that each monitor 30 can perform voice communication. will be connected to.
Note that the time period during which voice communication is possible is preferably continued for a predetermined period of time (for example, about 30 seconds) even after the identification mark 1g of the other party's unmanned moving object 1 is no longer detected.

Further, when the voice communication control unit 29c controls the monitor 30 of the first unmanned moving body 1 and the monitor 30 of the second unmanned moving body 1 to enable voice communication, the screen display unit 23 Each player displays the other's monitor screen as a pop-up on the display screen.
Specifically, as shown in FIG. 10, on the display screen of the monitor 30 of the first unmanned moving object 1, a monitor screen 39 with individual information (image of the operator, name) of the second unmanned moving object is displayed. will pop up.
Note that when the audio communication control unit 29c controls each monitor 30 to enable audio communication, it also controls the operator of each unmanned mobile object 1 to transmit dedicated BGM selected in advance to the other party. Also good. That is, it is preferable that the dedicated BGM of the second unmanned moving object 1 is played from the monitor 30 of the first unmanned moving object 1.
Furthermore, since the relative distance to the second unmanned moving object 1 can be identified based on the size of the detected identification mark 1g, the second unmanned moving object 1 is more sensitive to the first unmanned moving object 1. The exclusive BGM of the second unmanned moving object 1 may be made to play louder as it approaches.

In the above configuration, when the voice communication control unit 29c is executed, the second photographing device 1ab located behind the unmanned moving object 1 is used to detect the identification mark 1g of the other unmanned moving object 1 approaching from behind. can do. By doing so, it is possible to enhance the performance effect of the voice communication control section 29c.

The advertisement display control unit 29d controls the display of advertisements for the driver and viewers during the unmanned mobile race.
Specifically, while the viewer is viewing the race video on the mobile terminal 50, the advertisement display control unit 29d displays an advertisement corresponding to the viewer's attributes superimposed on a predetermined position of the race video.
Specifically, as shown in FIG. 11, an advertisement 51 regarding the next date of the drone race in which the viewer is interested is displayed in the lower left portion of the switching display screen.
Alternatively, while the driver is checking the race video on the monitor 30, an advertisement corresponding to the driver's attributes may be displayed superimposed on a predetermined position (for example, on the course side) of the race video.

The ghost display control unit 29e displays a virtual image ( (ghost images) are displayed on top of each other.
To be more specific, the ghost image can be displayed in an expanded manner based on the information on the fastest lap time and the average lap time of the operator who operates the unmanned moving object 1.
Specifically, as shown in FIG. 12, a virtual image (ghost image G) of the unmanned moving object 1 and individual ghost information G1 are expanded in real time on the real image taken by the unmanned moving object 1. Displayed.
Note that when displaying the virtual image (ghost image), by increasing the number of external photographing devices 60 and passing gates 70 installed on the course, the display of the virtual image can be accurately reproduced.

<Video processing method>
Next, the processing of the main video processing program (video processing method) executed by the video processing device 20 will be described based on FIG. 13.
The program according to the present embodiment includes the above-mentioned video data acquisition section 22, screen display section 23, mark detection section 24, and mobile object as functional components of the video processing device 20 including the storage section 21. A utility program that integrates various programs to realize the identification section 25, the gate passage determination section 26, the calculation section 27, the trouble processing section 28, and the mobile object control section 29, and is a utility program that integrates various programs for the video processing device. Twenty CPUs execute this video processing program.
Note that the above program is executed upon receiving an operation from the user to start video processing.

The "video processing flow" shown in FIG. 13 begins with step S1 in which the video data acquisition unit 22 acquires "video data" from each unmanned moving object 1.
Note that the acquired video data is classified for each unmanned moving object 1 and stored in the storage unit 21.

Next, in step S2, the screen display unit 23 (video display unit 23a) displays the video (real video) indicated by the “video data” acquired from each unmanned mobile object 1 on the monitor 30, display 40, and mobile terminal 50. indicate.

Next, in step S3, the mark detection unit 24 detects the presence of the identification mark 1g to be detected in the video indicated by the "video data" acquired from the first unmanned moving object 1, for example.
If the mark detection unit 24 detects that the identification mark 1g is present in the video (step S3: Yes), the process proceeds to step S4. On the other hand, if the identification mark 1g does not exist in the video (step S3: No), the process advances to step S9.

Next, in step S4, the mobile object identification unit 25 identifies which unmanned mobile object the unmanned mobile object 1 provided with the detected identification mark 1g is.
For example, the mobile object identification unit 25 identifies, in the video shot by the first unmanned mobile object 1, that the unmanned mobile object provided with the detected identification mark 1g is the second unmanned mobile object 1. do.

Next, in step S5, the mark detection section 24 and the moving object identification section 25 determine whether a plurality of identification marks 1g of the identified second unmanned moving object 1 have been detected.
Note that since the identification marks 1g are attached to the top, bottom, front, back, left and right of the unmanned moving body 1, a plurality of identification marks 1g of the second unmanned moving body 1 may be detected in the above video.
If the mobile object identification unit 25 detects a plurality of identification marks 1g of the second unmanned mobile object 1 (step S5: Yes), the process proceeds to step S6. On the other hand, if a plurality of identification marks 1g are not detected (step S5: No), the process advances to step S7.

Next, in step S6, the mobile object identification unit 25 selects the largest size identification mark 1g among the plurality of detected identification marks 1g of the second unmanned mobile object 1.
By selecting the identification mark 1g with the largest size, the accuracy of the relative position (relative distance) of the second unmanned moving body 1 with respect to the first unmanned moving body 1 can be improved.

Next, in step S7, the video processing device 20 (information acquisition unit) acquires the individual information and relative position information of the second unmanned moving object 1 identified by the identification mark 1g.
Here, "individual information" refers to information on the operator of the second unmanned vehicle 1 (operator's image, operator's name), as well as information on the fastest speed of the second unmanned vehicle 1 that is updated during the race. This includes information on the lap time (FT), the current ranking during the race, and the time difference (Behind) with the first unmanned moving object 1.
“Relative position information” corresponds to information on the relative position (relative distance) of the second unmanned moving body 1 with respect to the first unmanned moving body 1. For example, according to the display screen in FIG. 7, it can be seen that the second unmanned moving body 1 is located diagonally to the left in front of the first unmanned moving body 1, and the relative distance of the second unmanned moving body 1 can be calculated. It is also possible to calculate.

Next, in step S8, the screen display section 23 (virtual information display section 23b) displays the target position (relative position of the second unmanned moving object 1) on the video shot by the first unmanned moving object 1. The individual information of the second unmanned moving object 1 is overlaid and displayed in an expanded manner.
Looking at the display screen of FIG. 7, the individual information of the second unmanned moving object 1 (operator's name, fastest lap time ( FT) and time difference (Behind)) are displayed in an overlapping manner.

Finally, if an operation to stop the video processing program is received from the user (step S9: Yes), the process of FIG. 13 is ended. If the video processing program is to be continued (step S9: No), the process returns to step S1.
The processing flow of the video processing program described above makes it possible to provide higher entertainment value when operating an unmanned mobile race.
It will also be possible to provide a new experience that combines the real world and virtual world.

<Other embodiments>
In the above embodiment, as shown in FIG. 1, the unmanned moving object 1 is a small RC car, but it is not particularly limited to an RC car, and can be changed as appropriate as long as it is an unmanned moving object equipped with a photographing device. It is.
For example, it may be an unmanned aerial vehicle (drone), an unmanned helicopter flying in the air, a ship or a yacht moving on water, or the like. Furthermore, the present invention is not limited to toys, and can be widely applied to commercial unmanned aircraft, unmanned cars, and the like. Further, for example, it may be a manned moving body.

In the above embodiment, as shown in FIG. 1, the video processing system S is a system for operating an unmanned mobile race, but is not particularly limited to a system for an unmanned mobile race; It can be widely applied to various businesses as a video processing system or video processing device using an unmanned moving object equipped with a .

In the above embodiment, as shown in FIG. 1, a plurality of unmanned moving objects 1 are used in the present video processing system S, but there is no particular limitation, and if used as a commercial system, for example, The number of unmanned moving objects may be one.

In the above embodiment, as shown in FIGS. 3 and 4A to 4C, the identification mark 1g and the second identification mark 70A are two-dimensional barcodes, but are not particularly limited to marks that can be detected in the video. If so, it is widely applicable. Preferably, the identification mark is a mark capable of storing identification data.

In the above embodiment, as shown in FIG. 3, the identification marks 1g are attached to the top, bottom, front, back, left, and right outer surfaces of the unmanned vehicle 1, but the arrangement pattern of the identification marks 1g is not particularly limited. It can be changed as appropriate.
For example, the identification mark 1g may be attached to two places, the front part (front face) and the rear part (rear face) of the unmanned moving body 1, or it may be attached only to one place, the rear part of the unmanned moving body 1. good.

In the above embodiment, as shown in FIG. 7, the screen display unit 23 (virtual information display unit 23b) displays the operator's name, fastest lap time (FT), time difference, etc. as individual information of the second unmanned moving object 1. (Behind) is expanded and displayed, the individual information of the second unmanned moving object 1 is not limited to these, and can be changed.
For example, information such as the model name of the second unmanned moving object 1, the image and battle record of the operator, and the current ranking may be displayed in an expanded manner.

In the embodiment described above, as shown in FIG. 9, the screen display unit 23 displays a switching display screen on which the entire course map is displayed as a display screen for the viewer, but there is no particular limitation. The switching display screen may be displayed on the display screen for the user.
Alternatively, the video processing device 20 may separately control a drone (not shown), and display the aerial footage taken by the drone on a display screen for the operator and viewers.

In the above embodiment, as shown in FIG. 7, the screen display section 23 displays "contents based on the determination result" when the gate passage determination section 26 determines that each unmanned mobile object 1 has passed through a predetermined passage gate 70. is displayed on the display screen.
At this time, the "content based on the determination result" is not particularly limited to information regarding the elapsed time and current position of each unmanned mobile object 1, but other information obtained from the above determination result, i.e., during the unmanned mobile object race. It may also include a wide range of other real-time information.
For example, if the unmanned moving object 1 successfully travels through the central part of the passage area of a predetermined passage gate 70, or if it deviates from the course and passes through a passage gate 70 other than the passage gate 70 that it should have passed through. It is also possible to display predetermined performance contents, etc.

In the embodiment described above, a video processing program is stored in a recording medium that can be read by the video processing device 20, and processing is executed by the video processing device 20 reading and executing the program. Here, the recording medium that can be read by the video processing device 20 refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like.
Further, the video processing program may be distributed to a user terminal (not shown) via a communication line, and the user terminal itself that receives the distribution may function as a video processing device and execute the program.

In the above embodiment, the video processing system, video processing method, and video processing device using an unmanned moving object according to the present invention have been mainly described.
However, the embodiments described above are merely examples for facilitating understanding of the present invention, and do not limit the present invention. The present invention may be modified and improved without departing from its spirit, and it goes without saying that the present invention includes equivalents thereof.

S Image processing system 1 Unmanned mobile object (radio control car)
1a Photographing device 1aa First photographing device 1ab Second photographing device 1ac Third photographing device 1ad Fourth photographing device 1b Transmitting/receiving antenna 1c Mobile unit 1d Drive unit 1e Processor 1f Battery 1g Identification mark 1h Detection sensor 2 Storage unit 3 Operation data receiving unit 4 Video data transmitting section 10 Operating terminal 20 Video processing device 21 Storage section 22 Video data acquisition section 23 Screen display section 23a Video display section 23b Virtual video display section 23c Elapsed time display section 23d Current position display section 24 Mark detection section 25 Mobile object Identification part
26 Gate passage determination unit 27 Calculation unit 27a Elapsed time calculation unit 27b Current position calculation unit 28 Trouble processing unit 28a Inoperability determination unit 28b Inoperability response unit 29 Mobile object control unit 29a Support control unit 29b Penalty control unit 29c Voice communication control unit 29d Advertisement display control unit 29e Ghost display control unit 30 Monitor (head mounted display)
31 "FPV" operation button 32 "TPV" operation button 33 "BEV" operation button 34 Individual information of the first unmanned moving object 35 Individual information of the second unmanned moving object 36 Overall race information 37 Individual race information 38 Course map 38a , 38b Position display icon 39 Monitor screen 40 Display 41 Operation button 42 Operation button 43 Overall race information 44 Course map 44a, 44b Position display icon 50 Mobile terminal 51 Advertisement 60 External photographing device 70, 170, 270 Passing gate 70A Second identification mark 71 Gate leg portion 72 Gate connection portion 271 Louver 80 Rescue drone G Ghost image G1 Ghost individual information

Claims (9)

Multiple unmanned vehicles equipped with imaging devices that move while filming external images,
A video processing system, comprising: a video processing device connected to the unmanned moving body via wireless communication and processing images taken by the unmanned moving body;
The video processing device includes:
a video data acquisition unit that acquires video data indicating a video shot by the unmanned mobile object;
a screen display unit that displays a video indicated by the acquired video data on a display screen;
a mark detection unit that detects the presence of an identification mark to be detected in a video indicated by the acquired video data;
a mobile object identification unit that identifies which unmanned mobile object the unmanned mobile object provided with the detected identification mark is when the identification mark is detected;
When a predetermined identification mark is detected in the video shot by the first unmanned moving object, and the unmanned moving object provided with the predetermined identification mark is identified as the second unmanned moving object,
The screen display section includes:
Displaying an image taken by the first unmanned mobile object,
A video processing system using an unmanned moving object, characterized in that information regarding the second unmanned moving object is displayed at a target position on the video. The unmanned mobile object is remotely controlled by an operating terminal and moves along a predetermined course,
The mark detection unit detects the presence of a second identification mark to be detected in the video indicated by the acquired video data,
When the second identification mark is detected in the image, the video processing device determines that the unmanned mobile object has passed through a passage gate provided with the detected second identification mark. further comprising a determination section;
When the mark detection unit detects the presence of the second identification mark provided at a predetermined passage gate in the video shot by the first unmanned moving object,
2. The screen display unit according to claim 1, wherein the screen display unit displays the image and also displays content based on a determination result that the first unmanned moving object has passed through the predetermined passage gate. An image processing system using the unmanned moving object described above. The unmanned moving body is
a first photographing device that is attached to a front position or a central position of the unmanned moving body and photographs a first-person video from a first-person viewpoint from the unmanned moving body;
A second photographing device is installed at a rear position of the unmanned moving body or at a position rearward of the main body of the unmanned moving body and shoots a third-person video from a third-person perspective including an image of the unmanned mobile body itself. ,
The unmanned vehicle according to claim 1 or 2, wherein the screen display unit receives a selection by a user operation and switches and displays the first-person video and the third-person video captured by the unmanned mobile object. An image processing system that uses the body. The unmanned mobile object is remotely controlled by an operating terminal and moves along a predetermined course,
further comprising a plurality of external photographing devices installed at predetermined positions on the course and photographing the unmanned moving body moving on the course,
The video data acquisition unit acquires video data indicating a video shot by the external imaging device,
Claims 1 to 3, wherein the screen display unit receives a selection by a user operation and switches and displays the images taken by the plurality of unmanned moving objects and the plurality of external photographing devices. A video processing system using the unmanned moving object according to any one of the above. The unmanned mobile object is remotely controlled by an operating terminal and moves along a predetermined course,
The video processing device includes:
an inoperability determination unit that determines that the unmanned mobile object provided with the detected identification mark is inoperable by remote control when the identification mark is detected in the video under predetermined conditions;
The method further includes an inoperability response unit that performs a predetermined response to the unmanned moving body in order to recover the unmanned moving body from the remote control disabled state when it is determined that the unmanned moving body is not remotely operable. An image processing system using an unmanned moving object according to any one of claims 1 to 4. The unmanned mobile object is remotely controlled by an operating terminal and moves along a predetermined course,
The video processing device further includes a mobile body control unit that performs predetermined response control for the plurality of unmanned mobile bodies in order to move the plurality of unmanned mobile bodies on a course under predetermined conditions,
Any one of claims 1 to 5, wherein the moving body control unit controls the moving operation of the plurality of unmanned moving bodies in order to cause the plurality of unmanned moving bodies to start moving at a predetermined timing. An image processing system using the unmanned moving object described in 2. The unmanned moving body is
It is remotely controlled by a control terminal and moves along a predetermined course.
further comprising a detection sensor provided on the front and rear surfaces of the unmanned moving body and detecting that a target object has contacted the unmanned moving body,
The video processing device further includes a mobile body control unit that performs predetermined response control for the plurality of unmanned mobile bodies in order to move the plurality of unmanned mobile bodies on a course under predetermined conditions,
The mobile body control unit is characterized in that when receiving a detection signal from the detection sensor from a predetermined unmanned mobile body, the mobile body control unit controls to limit or stop the movement operation of the predetermined unmanned mobile body for a predetermined period of time. An image processing system using the unmanned moving object according to any one of claims 1 to 6. A video processing method in which a computer connected by wireless communication to a plurality of unmanned mobile bodies equipped with a photographing device and moving while photographing external images processes images photographed by the unmanned mobile bodies, the method comprising:
The computer,
a video data acquisition step of acquiring video data indicating a video shot by the unmanned mobile object;
a first screen display step of displaying a video indicated by the acquired video data on a display screen;
a mark detection step of detecting the presence of an identification mark to be detected in the video indicated by the acquired video data;
when the identification mark is detected, a moving object identification step of identifying which unmanned moving object the unmanned moving object provided with the detected identification mark is;
In the mark detection step, a predetermined identification mark is detected in the video shot by the first unmanned moving object, and in the moving object identification step, the unmanned moving object provided with the predetermined identification mark is detected as a second unmanned moving object. When identifying that it is an unmanned moving object,
The present invention is characterized by further executing a second screen display step of displaying an image taken by the first unmanned moving object and displaying information regarding the second unmanned moving object at a target position on the image. Image processing method using unmanned moving objects. An image processing device equipped with a photographing device, connected by wireless communication to a plurality of unmanned moving bodies that move while photographing external images, and processing images photographed by the unmanned mobile bodies,
a video data acquisition unit that acquires video data indicating a video shot by the unmanned mobile object;
a screen display unit that displays a video indicated by the acquired video data on a display screen;
a mark detection unit that detects the presence of an identification mark to be detected in a video indicated by the acquired video data;
a mobile object identification unit that identifies which unmanned mobile object the unmanned mobile object provided with the detected identification mark is when the identification mark is detected;
When a predetermined identification mark is detected in the video shot by the first unmanned moving object, and the unmanned moving object provided with the predetermined identification mark is identified as the second unmanned moving object,
An image using an unmanned moving object, wherein the screen display unit displays information regarding the second unmanned moving object superimposed on a target position on the image taken by the first unmanned moving object. Processing equipment.

Images