JP6840441B2 - Imaging system and flying object - Google Patents

Description

The present invention relates to an imaging system and an air vehicle in which an imaging object or the like is simultaneously imaged from a plurality of directions.

Conventionally, imaging is performed by each of the first and second imaging devices and recorded together with audio data, time lag information is calculated based on these two audio data, and the reproduction timing of each captured image is shifted by this time lag. An image synchronization system is known in which two images are synchronized by controlling based on information (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2008-193561

By the way, in the above-mentioned Patent Document 1, audio data is recorded corresponding to the imaging operation in the first imaging device, and audio data is recorded corresponding to the imaging operation in the second imaging device. There is no description about the source of audio data. Since each of the first and second image pickup devices is provided with a sound reproduction unit and a sound collection unit, it is conceivable that each of the first and second image pickup devices separately generates sound and collects sound. However, in that case, there is a problem that it is difficult to accurately synchronize a plurality of images based on the separately generated sounds. Further, in the image synchronization system of Patent Document 1, each of the first and second image pickup devices requires a voice synchronization detection unit, a communication unit for transmitting and receiving voice data and the like to and from each other, and the configuration and processing are required. It is complicated, and there is a problem that it is not possible to take an image of one subject using a general-purpose camera and synchronize the images.

The present invention was created in view of these points, and an object of the present invention is that the configuration is simple, the processing is not complicated, and a plurality of images captured separately can be easily synchronized. The purpose is to provide an imaging system and an air vehicle.

In order to solve the above-mentioned problems, the image pickup system of the present invention includes an air vehicle equipped with a first image pickup means and an image pickup device having a second image pickup means. The flying object is output from a flight control means for causing the flying object to fly around a predetermined imaging object, a sound wave output means for outputting a sound wave having a predetermined frequency having an output level reachable to the imaging device, and a sound wave output means. A first sound collecting means for collecting sound waves and a first imaging means for capturing an image of an object to be imaged and recording the sound waves collected by the first sound collecting means together with the image obtained by this imaging. And. Further, the image pickup apparatus captures a second sound collecting means for collecting sound waves output from the sound wave output means, an image pickup target and / or a flying object, and a second image obtained by the image pickup. A second imaging means for recording the sound waves collected by the sound collecting means is provided. After a predetermined time has elapsed since the recording of the image and the sound wave by the first imaging means is started, when the operator instructs the output of the sound wave, the sound wave is repeated every time the predetermined time elapses. A predetermined sound wave is output from the output means.

The first imaging means and the first sound collecting means mounted on the flying object, and the second imaging means and the second sound collecting means provided in the imaging device can be easily realized by using a general-purpose video camera. Therefore, it is possible to construct the entire system simply by adding the sound wave output means to the flying object, and it is possible to simplify the configuration and the processing contents. In addition, since the sound waves output from the sound wave output means of the flying object are recorded at the same time as the images in each of the flying object and the image pickup device, a plurality of images separately captured should be synchronized according to the sound waves. Becomes easier.

Further, it is desirable that the sound wave output from the above-mentioned sound wave output means includes specific information indicating the content of imaging by the first imaging means. This facilitates the correspondence between the flying object and the sound wave, and it is possible to prevent the images from being synchronized due to an incorrect combination.

Further, it is desirable that the above-mentioned specific information includes at least one of the identification information of the flying object, the position information of the flying object, and the imaging time information by the first imaging means. When a plurality of images are captured by the same combination of the flying object and the imaging device, it becomes easy to synchronize the respective images.

Further, it is desirable that the sound wave output from the above-mentioned sound wave output means uses a frequency component other than the human audible frequency band. This makes it possible to prevent hearing annoying sounds during shooting.

Further, it is desirable that the sound wave output from the above-mentioned sound wave output means uses a frequency component included in the human audible frequency band. As a result, the user can directly confirm the presence or absence of the sound wave output for synchronizing the images with his / her ear, and can quickly grasp the problem that the sound wave is not output.

Further, the flying object of the present invention is an flying object equipped with an imaging means, and is located at a position away from the flying object and a flight control means for causing the flying object to fly around a predetermined imaging object. A sound wave output means that outputs a sound wave of a predetermined frequency having an output level that can reach an image pickup device different from the body, a sound wave collection means that collects the sound wave output from the sound wave output means, and an image pickup object are imaged. Along with the image obtained by this imaging, the imaging means for recording the sound waves collected by the sound collecting means is provided. When a predetermined time elapses after the recording of the image and the sound wave by the imaging means is started, when the operator instructs the output of the sound wave, the sound wave output means repeats each time the predetermined time elapses. A predetermined sound wave is output.

Since the imaging means and sound collecting means mounted on the flying object can be easily realized by using a general-purpose video camera, the flying object can be configured simply by adding the sound wave output means. And it is possible to simplify the processing contents. Further, since the sound wave output from the sound wave output means of the flying object is recorded at the same time as the image in each of the flying object and another video camera or the like, synchronization of a plurality of images separately captured in accordance with this sound wave. It becomes easy to take.

It is a figure which shows the outline of the image pickup system of one Embodiment. It is a figure which shows the structure of a drone. It is a flow chart which shows the operation procedure which image | imaged the transmission line as an image | image object using the video camera mounted on the drone. It is a flow chart which shows the operation procedure which image | imaged the drone which is image | imaging the transmission line using the video camera which the operator A possesses. It is a figure which shows the image-taking and recording contents by two video cameras.

Hereinafter, an imaging system according to an embodiment to which the present invention is applied will be described with reference to the drawings.

FIG. 1 is a diagram showing an outline of an imaging system of one embodiment. As shown in FIG. 1, the imaging system of the present embodiment includes a drone (unmanned air vehicle) 100 as an air vehicle equipped with a video camera 200 and a video camera 300 operated by an operator A on the ground. It is composed including and.

For example, the video camera 200 mounted on the drone 100 is for checking the presence or absence of a failure of the power transmission line 400 by imaging a power transmission line as an image pickup target when a lightning strike occurs and analyzing the captured image. .. Further, the video camera 300 possessed by the operator A is for confirming that there is no abnormality in the imaging environment by imaging the drone 100 including the video camera 200 that is imaging the transmission line.

FIG. 2 is a diagram showing the configuration of the drone 100. As shown in FIG. 2, the drone 100 includes a communication unit 110, a flight instruction receiving unit 120, a sound wave output unit 122, a speaker 124, a flight control unit 130, an attitude control unit 140, a drive control unit 142, a motor 144, and a video camera 200. Is configured to include.

The communication unit 110 wirelessly connects to, for example, the remote control (remote control) device 150 held by the operator B to transmit and receive data.

The flight instruction receiving unit 120 receives various instructions for remote control sent from the remote controller 150 by wireless communication via the communication unit 110.

The sound wave output unit 122 generates a sound wave having a predetermined frequency having an output level that can reach the video camera 300 held by the operator A on the ground and outputs the sound wave from the speaker 124. The sound wave output from the sound wave output unit 122 includes specific information indicating the content captured by the video camera 200. For example, as this specific information, at least one of the identification information of the drone 100, the position information of the drone 100, and the image pickup time information by the video camera 200 is included.

The sound wave output unit 122 has an output data generation unit 122a and a modulation unit 122b. The output data generation unit 122a generates output data including all or part of the identification information of the drone 100, the position information, the imaging time by the video camera 200, the operating state of the drone 100, and the like. The modulation unit 122b modulates the generated output data, superimposes it on a sound wave having a predetermined frequency, and outputs it from the speaker 124. The sound wave output from the speaker 124 may use a frequency component other than the human audible frequency band, or may use a frequency component included in the human audible frequency band. In the former case where a band other than the audible frequency band is used, it is possible to prevent harsh sounds from being heard during shooting. Further, in the latter case where the audible frequency band is used, the user can directly confirm by ear the presence or absence of sound waves used when synchronizing the images obtained by imaging the two video cameras 200 and 300, respectively. It is possible to quickly grasp the problem that sound waves are not output.

The flight control unit 130 performs flight control for causing the drone 100 to fly around the object to be imaged (“transmission line” in the example shown in FIG. 1). The attitude control unit 140 includes a GPS receiver and a gyro sensor, detects the position of the drone 100, and controls the attitude (position and orientation). The position of the drone 100 detected by the attitude control unit 140 is sent to the output data generation unit 122a of the sound wave output unit 122 and used as "position information" included in the output data.

The drive control unit 142 drives a plurality of (for example, four) motors 144 to rotate a plurality of propellers (not shown) corresponding to each, thereby causing the drone 100 to fly in a predetermined direction at a predetermined speed. ..

The video camera 200 described above is used as the first imaging means and the first sound collecting means, the video camera 300 is used as the second imaging means and the second sound collecting means, the image pickup device is used, and the flight control unit 130 is used as the flight control means. The sound wave output unit 122 corresponds to the sound wave output means, respectively.

The imaging system of the present embodiment has such a configuration, and next, an operation of imaging a transmission line as an imaging object will be described.

FIG. 3 is a flow chart showing an operation procedure for imaging a power transmission line as an imaging object using a video camera 200 mounted on the drone 100.

Prior to the start of imaging, the operator B operates the remote controller 150 to instruct the drone 100 to fly. When this instruction is received by the flight instruction receiving unit 120 in the drone 100 via the communication unit 110, the flight control unit 130 flies the drone 100 to the imaging position of the transmission line as an imaging object in response to this instruction ( Step 100). Instead of the operator B operating the remote control device 150 to remotely control the drone 100 to fly to the imaging position, the operator B sent the imaging position information to the drone 100 and performed autonomous flight under the control of the flight control unit 130. The drone 100 may be moved to the imaging position.

Next, when the operator B operates the remote controller 150 to send an imaging start instruction to the drone 100 and the flight instruction receiving unit 120 receives the imaging start instruction via the communication unit 110, the video camera 200 sends a power transmission line. Recording of the ambient sound is started at the same time as the imaging of (step 102). In the present embodiment, the video camera 200 can be a commercially available product, and the recording operation is performed using the built-in microphone (microphone) in parallel with the imaging operation, but an external microphone is used. Alternatively, the imaging portion, the recording portion, and the recording portion of the imaging / recording result may be prepared as separate parts, and these parts may be assembled when the drone 100 is assembled.

Next, the sound wave output unit 122 outputs a predetermined sound wave (step 104). The timing of outputting the sound wave is as follows: (1) Output after a predetermined time has elapsed from the start of imaging and recording in step 102, (2) The remote controller device 150 is operated by the operator B to instruct the output of the sound wave. It is conceivable that the output may occur occasionally, or (3) the output may be repeated every time a predetermined time elapses.

Next, when the operator B operates the remote controller 150 to send an imaging end instruction to the drone 100 and the flight instruction receiving unit 120 receives the imaging end instruction via the communication unit 110, the video camera 200 sends a power transmission line. And the recording of the ambient sound is finished (step 106).

After the imaging is completed, the operator B operates the remote controller device 150 to instruct the drone 100 to end the flight. When this instruction is received by the flight instruction receiving unit 120 in the drone 100 via the communication unit 110, the flight control unit 130 makes the drone 100 land at a predetermined landing position in response to this instruction and performs a series of imaging. End the flight (step 108). If an abnormality such as a crash occurs during the imaging / recording operation of the drone 100 by the video camera 200, the imaging / recording operation of the video camera 200 may be interrupted at that time. Therefore, the above-mentioned step 106 , 108 may not be performed.

FIG. 4 is a flow chart showing an operation procedure for imaging the drone 100 while imaging the transmission line by using the video camera 300 possessed by the operator A.

The operator A visually confirms, for example, that the drone 100 has started flying (step 200). After that, when the operation unit of the video camera 300 is operated by the operator A to instruct the start of imaging, the video camera 300 starts recording the ambient sound together with the imaging of the drone 100 that is imaging the transmission line (step 202). ). Since the volume of the sound wave output from the speaker 124 of the drone 100 is set so as to reach the operator A, the sound wave output from the speaker 124 is recorded by the video camera 300 in parallel with the imaging.

Next, the operator A visually confirms, for example, that the drone 100 has completed the flight (step 204). After that, when the operation unit of the video camera 300 is operated by the operator A to instruct the end of imaging, the video camera 300 ends the imaging of the drone 100 and the recording of the ambient sound (step 206). In this way, a series of imaging and recording operations by the video camera 300 are completed.

FIG. 5 is a diagram showing the contents of imaging and recording by the two video cameras 200 and 300. In FIG. 5, "Flying Log" is the event content (operating state) of the drone 100, and "UAV (Unmanned aerial vehicle) recording" is the recorded content (imaging, recording content) of the video camera 200 mounted on the drone 100. “Ground recording” indicates the recorded contents of the video camera 300 that the operator A holds and operates.

In the example shown in FIG. 5, the drone 100 starts flying at time 00:40, reaches the imaging position at time 00:52, and starts imaging and recording with the video camera 200 at time 00:53. Further, when a voltage drop of the battery (not shown) mounted on the drone 100 is detected at time 00:55, the identification information of the drone 100 is included from the speaker 124 in parallel with this detection operation (at the same time). The sound wave is output for a predetermined time (the time required to output all the data). For example, in this sound wave, the message data "UAV No. 1 voltage drop at 00:55", which is a combination of the detection of the battery voltage drop, the detection time of 00:55, and the aircraft number as the identification information of the drone 100, is displayed. It is modulated and included.

After that, after detecting the altitude drop at time 01:12, when it is detected that the vehicle is crashing at time 01:13, the identification information of the drone 100 is detected from the speaker 124 in parallel with this detection operation (at the same time). The sound wave including the above is output again for a predetermined time. For example, this sound wave contains modulated message data that combines the detection that the drone 100 is crashing, the detection time 01:13, and the aircraft number as the identification information of the drone 100. ing.

In the video camera 200 mounted on the drone 100, a sound wave output from the speaker 124 for a predetermined time from time 00:55 and a sound wave output from the speaker 124 for a predetermined time from time 01:13 are generated together with the moving image obtained by imaging. Recorded.

Similarly, in the video camera 300 held by the operator A, the sound wave output from the speaker 124 for a predetermined time from time 00:55 and the sound wave output from the speaker 124 from time 01:13 are predetermined together with the moving image obtained by the imaging. The time output sound wave is recorded. Since the imaging start time by the video camera 300 does not match the imaging start time by the video camera 200 mounted on the drone 100, the time from the start of recording to the first sound wave output time 00:55 is different.

When it becomes necessary to analyze these two types of recorded images (for example, when investigating the situation of the crash of the drone 100), the sound waves contained in each recorded content are demodulated and examined to obtain the sound waves. It is possible to specify the recording content and the recording start time (time 00:55 and time 01:13), and it is possible to synchronize the images recorded by the video cameras 200 and 300 with reference to the recording start time. It becomes.

This synchronization may be performed manually by an analyst who analyzes the recorded image, or may be performed automatically using an analysis device. In the case of manual operation, the reproduction is temporarily interrupted at the timing (time 00:55) when one of the recorded images is reproduced and the sound wave is heard. Similarly, the other recorded image is reproduced and the reproduction is temporarily interrupted at the timing when the sound wave is heard. After that, by restarting the reproduction of these two types of recorded images at the same time, it is possible to reproduce the two types of recorded images in a synchronized state. In the case of automatic operation, the analysis device may perform the same processing as in the case of manual operation.

As described above, in the imaging system of the present embodiment, the video camera 200 mounted on the drone 100 and the video camera 300 held by the operator A can be easily realized by using a general-purpose product. Therefore, the entire system can be constructed only by adding the sound wave output unit 122 to the drone 100, and the configuration and the processing contents can be simplified. Further, since the sound wave output from the sound wave output unit 122 of the drone 100 is recorded at the same time as the image in each of the video camera 200 of the drone 100 and the video camera 300 held by the operator A, the sound wave is matched with this sound wave. Therefore, it becomes easy to synchronize these images captured separately.

Further, the sound source output from the sound wave output unit 122 of the drone 100 includes specific information indicating the content captured by the video camera 200, specifically, the identification information of the drone 100, the position information of the drone 100, and the imaging time by the video camera 200. Contains at least one piece of information. As a result, it becomes easy to make a correspondence between the drone 100 and the sound wave, and it is possible to prevent the images from being synchronized by an erroneous combination. Further, by including such specific information in the sound wave, a plurality of images (moving images) are captured by the same combination of the video camera 200 mounted on the drone 100 and the video camera 300 held by the operator A. Even if there is, it becomes easy to synchronize each image.

The present invention is not limited to the above embodiment, and various modifications can be made within the scope of the gist of the present invention. For example, in the above-described embodiment, one video camera 200 mounted on the drone 100 images a transmission line as an imaging object, and the other video camera 300 held by the operator A images the drone 100. However, the video camera 300 may be used to image the transmission line, which is the same object to be imaged. Alternatively, with multiple video cameras mounted on each of the multiple drones, or with one or more video cameras held by each of the multiple operators or fixed to the ground or a building, etc. May be arbitrarily combined to image an imaging object or a drone. In that case, the sound wave output unit 122 and the speaker 124 may be mounted on at least one drone, and sound waves including specific information may be output from one drone.

Further, in the above-described embodiment, when an abnormality occurs in the drone 100, the audio output unit 122 outputs a sound wave including specific information from the speaker 124. However, regardless of the presence or absence of the abnormality, for example, the drone 100 The sound wave may be output immediately after reaching a predetermined position and starting imaging (or after a certain period of time has elapsed, or after that, a fixed period or an indefinite period may be obtained).

Further, in the above-described embodiment, in the drone 100, the content obtained by modulating the output data generated by the output data generation unit 122a by the modulation unit 122b is superimposed on the output voice, but the output data (specific information) is superimposed on the output sound wave. Any method may be used for the method. For example, the output data generation unit 122a may generate PCM data indicating a voice that reads out the contents of the specific information, and the PCM data may be digital-to-analog converted and output from the speaker 124.

As described above, according to the present invention, the first imaging means, the first sound collecting means, the second imaging means, and the second sound collecting means provided in the imaging device are Since it can be easily realized using a general-purpose video camera, the entire system can be constructed simply by adding sound wave output means to the flying object, and the configuration and processing contents can be simplified. Become. In addition, since the sound waves output from the sound wave output means of the flying object are recorded at the same time as the images in each of the flying object and the image pickup device, a plurality of images separately captured should be synchronized according to the sound waves. Becomes easier.

100 drone (unmanned aerial vehicle)
110 Communication unit 120 Flight instruction receiver 122 Sound wave output unit 122a Output data generation unit 122b Modulation unit 124 Speaker 130 Flight control unit 140 Attitude control unit 142 Drive control unit 144 Motor 200, 300 Video camera 400 Transmission line

Claims (6)

In an imaging system including an air vehicle equipped with a first imaging means and an imaging device having a second imaging means.
The flying object
A flight control means for causing the flying object to fly around a predetermined imaging object, and
A sound wave output means for outputting a sound wave having a predetermined frequency having an output level that can reach the image pickup device, and a sound wave output means.
A first sound collecting means for collecting sound waves output from the sound wave output means, and
The first imaging means for imaging the object to be imaged and recording the sound waves collected by the first sound collecting means together with the image obtained by the imaging is provided.
The image pickup device
A second sound collecting means for collecting sound waves output from the sound wave output means, and
The second imaging means, which captures the imaging object and / or the flying object and records the sound waves collected by the second sound collecting means together with the image obtained by the imaging, is provided .
After a predetermined time has elapsed from the start of recording the image and the sound wave by the first imaging means, when the operator instructs the output of the sound wave, the sound wave is repeated every time the predetermined time elapses, at any of the timings. imaging system characterized in Rukoto predetermined wave is output from the sound wave output unit. The imaging system according to claim 1, wherein the sound waves output from the sound wave output means include specific information indicating the content of imaging by the first imaging means. The imaging system according to claim 2, wherein the specific information includes at least one of the identification information of the flying object, the position information of the flying object, and the imaging time information by the first imaging means. The imaging system according to any one of claims 1 to 3, wherein the sound wave output from the sound wave output means uses a frequency component other than the human audible frequency band. The imaging system according to any one of claims 1 to 3, wherein the sound wave output from the sound wave output means uses a frequency component included in a human audible frequency band. An air vehicle equipped with imaging means
A flight control means for causing the flying object to fly around a predetermined imaging object, and
A sound wave output means that outputs a sound wave of a predetermined frequency that is located away from the flying object and has an output level that can reach an imaging device different from the flying object.
A sound collecting means for collecting sound waves output from the sound wave output means, and a sound collecting means.
The imaging means for imaging the object to be imaged and recording the sound waves collected by the sound collecting means together with the image obtained by the imaging.
When a predetermined time elapses after the recording of the image and the sound wave by the imaging means is started and the operator instructs the output of the sound wave, the sound wave is repeated every time the predetermined time elapses. aircraft, characterized in Rukoto predetermined wave is output from the sound wave output unit.

Images