JP7285927B2 - Control device - Google Patents

Description

The present invention relates to a technique for capturing an image of a subject using an aircraft.

A system has been proposed in which an unmanned aerial vehicle called a drone is used to take an image of a subject such as a power line and inspect the subject. In order to grasp the state of the subject from the captured image in this mechanism, it is necessary to capture the subject with the subject accurately focused from a short distance. For example, in Patent Document 1, when inspecting power transmission equipment with a camera mounted on a drone, it is possible to determine whether the subject is on the short side or the long side from the focus position based on the blur shape of the distance image. is disclosed.

JP 2019-9919 A

In the mechanism described in Japanese Patent Laid-Open No. 2002-200013, it is necessary to prepare in advance a distance image corresponding to each subject. SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to focus on a subject and perform imaging by a simpler method.

In order to solve the above-mentioned problems, the present invention provides size information about the size of an image of a subject displayed on a control device for controlling the aircraft when the aircraft captures an image of the subject from a position a predetermined distance away. and a control unit that causes the control device to display an image captured by the flying object, and causes the control device to display a guidance image having a size corresponding to the stored size information. wherein the control unit magnifies a partial area of an image captured by the aircraft by a certain magnification and causes the control device to display the image, wherein the subject is added to the image displayed by the control device is no longer included, the image captured by the flying object is displayed on the control device at a reduced magnification, and the control unit reduces the image captured by the flying object to the enlarged image When the image of the subject comes to be included in a partial area of the image picked up by the aircraft after the image is displayed on the control device at a reduced magnification, the enlargement magnification is increased and the image is displayed. Provided is a control device characterized in that, when displaying on a control device, an image captured by the flying object is displayed on the control device at the enlargement magnification corresponding to the number of times the enlargement magnification is changed per unit time. do.

Further, the present invention provides a storage unit for storing size information relating to the size of an image of a subject to be displayed on a display device when the flying vehicle captures an image of the subject from a position separated by a predetermined distance, and the flying vehicle. a control unit for causing the flying object to fly at a position such that the captured image of the subject is displayed on the display device in a size corresponding to the stored size information; A condition that makes it difficult for a viewer to identify the image of the subject at that time, and a condition that stores an image of a structure other than the subject in the background of the image of the subject when the subject is captured. and a storage unit, wherein the control unit enlarges a partial area of an image captured by the flying vehicle by a certain magnification and displays it on the display device, and the subject is a part of the image. A control characterized by controlling the flight of the flying object or the direction of imaging by the flying object so as to fall within the area of and further controlling the flying object so as not to meet the conditions stored in the storage unit Provide equipment.

The control unit causes the control device to display an image captured by the flying object, and controls the control device to display a guide image having a size corresponding to the stored size information; and controlling the flying object to fly to a position where the image of the subject imaged by is displayed on a display device in a size corresponding to the stored size information. .

According to the present invention, it is possible to take an image while focusing on a subject by a simpler method.

It is a figure showing an example of composition of flight system 1 concerning a 1st embodiment for carrying out the present invention. It is a figure which shows the hardware constitutions of the flying object 10 which concerns on 1st Embodiment. It is a figure which shows the hardware constitutions of the server apparatus 20 which concerns on 1st Embodiment. 3 is a diagram showing the hardware configuration of a control device 30 according to the first embodiment; FIG. It is a figure showing an example of functional composition of flight system 1 concerning a 1st embodiment. 4 is a flow chart showing an example of the operation of the server device 20 according to the first embodiment; 4 is a diagram showing an example of an image displayed on a control device 30; FIG. 3 is a diagram illustrating the relationship between an image captured by the flying object 10 and an image displayed on the control device 30; FIG. 3 is a diagram illustrating the relationship between an image captured by the flying object 10 and an image displayed on the control device 30; FIG. It is a figure which shows an example of a structure of the flight system 1a which concerns on 2nd Embodiment of this invention. It is a figure showing an example of functional composition of flight system 1a concerning a 2nd embodiment. 9 is a flow chart showing an example of the operation of the server device 20a according to the second embodiment;

[First embodiment]
[composition]
FIG. 1 is a diagram showing an example configuration of a flight system 1 according to a first embodiment for carrying out the present invention. The flight system 1 is a system for remotely inspecting an object to be inspected, such as a power line. A flight system 1 includes an unmanned flying object 10 called a drone, a server device 20, a control device 30 operated by an operator to control the flying object 10, and a communication network 2 that communicably connects them. Prepare. Server device 20 functions as a control device that controls display processing in control device 30 . The communication network 2 is, for example, a radio communication network such as LTE (Long Term Evolution).

The operator operates the control device 30 to operate the flying object 10 via the communication network 2, fly to a close range of the power transmission line as a subject, and make the flying object 10 image the power transmission line. The flying object 10 transmits imaged data of the transmission line to the server device 20 via the communication network 2, and the imaged data is accumulated in the server device 20. FIG. The inspector refers to the image indicated by the imaging data accumulated in the server device 20 and inspects whether the power transmission line is flawless. In addition, imaged data captured by the flying object 10 is transmitted to the control device 30 via the server device 20 , and an image indicated by the imaged data is displayed on the control device 30 . The operator steers the aircraft 10 while watching the image. In other words, the control device 30 functions as a display device that displays images captured by the flying object 10 . The inspector may refer to the image displayed on the control device 30 at this time to check whether the power transmission line is flawless.

FIG. 2 is a diagram showing the hardware configuration of the flying object 10. As shown in FIG. The aircraft 10 physically includes a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a flight device 1007, a sensor 1008, a positioning device 1009, and a bus connecting them. It is configured as a computer device. Each of these devices operates with power supplied from a battery (not shown). Note that in the following description, the term "apparatus" can be read as a circuit, device, unit, or the like. The hardware configuration of the aircraft 10 may be configured to include one or more of each device shown in the figure, or may be configured without some of the devices.

Each function in the flying object 10 is performed by causing the processor 1001 to perform calculations, controlling communication by the communication device 1004 and controlling the communication by the memory 1002 by loading predetermined software (programs) onto hardware such as the processor 1001 and the memory 1002 . and by controlling at least one of reading and writing of data in the storage 1003 .

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like. Also, for example, a baseband signal processing unit, a call processing unit, and the like may be implemented by the processor 1001 .

The processor 1001 reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 to the memory 1002, and executes various processes according to them. As the program, a program that causes a computer to execute at least part of the operations described later is used. The functional blocks of the aircraft 10 may be implemented by control programs stored in the memory 1002 and running on the processor 1001 . Various types of processing may be executed by one processor 1001, but may also be executed by two or more processors 1001 simultaneously or sequentially. Processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from the communication network 2 to the aircraft 10 via an electric communication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and RAM (Random Access Memory). may be The memory 1002 may also be called a register, cache, main memory (main storage device), or the like. The memory 1002 can store executable programs (program code), software modules, etc. to perform the methods of the present invention.

The storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like. Storage 1003 may also be called an auxiliary storage device. The storage 1003 stores, for example, identification information of the aircraft 10 (referred to as aircraft identification information). This flying object identification information is used by the server device 20 to identify and control the flying object 10 .

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via the communication network 2, and is also called a network device, a network controller, a network card, a communication module, or the like. The communication device 1004 includes a high-frequency switch, a duplexer, a filter, a frequency synthesizer, and the like, for example, in order to realize at least one of frequency division duplex (FDD) and time division duplex (TDD). may consist of For example, a transmitting/receiving antenna, an amplifier section, a transmitting/receiving section, a transmission line interface, etc. may be implemented by the communication device 1004 . The transmitting/receiving section may be physically or logically separated into a transmission control section and a receiving section.

The input device 1005 is an input device (for example, a key, a microphone, a switch, a button, etc.) that receives input from the outside, and particularly includes an imaging device that captures an image of a power transmission line as a subject. The output device 1006 is an output device (eg, display, speaker, LED lamp, etc.) that outputs to the outside.

The flight device 1007 is a mechanism for causing the aircraft 10 to fly in the air, and includes, for example, a propeller, a motor for driving the propeller, and a drive mechanism.

The sensor 1008 includes, for example, a temperature sensor, a rotation speed sensor that detects the rotation speed of a motor, a sensor that detects a value related to some input/output such as current/voltage (for example, a battery power remaining sensor), a gyro sensor, an acceleration sensor, It includes a group of sensors such as barometric pressure (altitude) sensors, magnetic (orientation) sensors, and ultrasonic sensors.

The positioning device 1009 measures the three-dimensional position of the flying object 10 . The positioning device 1009 is a GPS (Global Positioning System) receiver and measures the position of the aircraft 10 based on GPS signals received from a plurality of satellites.

Each device such as processor 1001 and memory 1002 is connected by a bus for communicating information. The bus may be configured using a single bus, or may be configured using different buses between devices.

The aircraft 10 includes hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). Alternatively, the hardware may implement part or all of each functional block. For example, processor 1001 may be implemented using at least one of these pieces of hardware.

FIG. 3 is a diagram showing the hardware configuration of the server device 20. As shown in FIG. The server device 20 is physically configured as a computer device including a processor 2001, a memory 2002, a storage 2003, a communication device 2004, an input device 2005, an output device 2006, and a bus connecting them. Each function in the server apparatus 20 is performed by causing the processor 2001 to perform calculations, controlling communication by the communication apparatus 2004, and controlling the and by controlling at least one of reading and writing data in the storage 2003 . A processor 2001, a memory 2002, a storage 2003, a communication device 2004, and a bus connecting them are the same as the processor 1001, memory 1002, storage 1003, communication device 1004, and a bus connecting these described for the aircraft 10. Since it is the same, the description thereof is omitted.

The input device 2005 is an input device (for example, keyboard, mouse, microphone, switch, button, sensor, joystick, ball controller, etc.) that receives input from the outside. The output device 2006 is an output device (for example, display, speaker, LED lamp, etc.) that outputs to the outside. Note that the input device 2005 and the output device 2006 may be integrated (for example, a touch panel).

FIG. 4 is a diagram showing the hardware configuration of the control device 30. As shown in FIG. The control device 30 is physically configured as a computer device including a processor 3001, a memory 3002, a storage 3003, a communication device 3004, an input device 3005, an output device 3006, and a bus connecting them. Each function of the control device 30 is performed by causing the processor 3001 to perform calculations, controlling communication by the communication device 3004, and controlling the communication by the memory 3002 by loading predetermined software (programs) onto hardware such as the processor 3001 and the memory 3002. and by controlling at least one of data reading and writing in the storage 3003 . A processor 3001, a memory 3002, a storage 3003, a communication device 3004, an input device 3005, an output device 3006, and a bus connecting them are the same as the processor 2001, memory 2002, storage 2003, communication device 2004, and input device 2005 described for the server device 20. , the output device 2006 and the bus connecting them are the same as the hardware, so description thereof will be omitted. The output device 3006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside, and particularly includes a display device that displays images.

In this embodiment, an example in which the flying object 10 and the control device 30 communicate via the communication network 2 will be described. you can go

FIG. 5 is a diagram showing an example of the functional configuration of the flight system 1. As shown in FIG. In the server device 20, the communication unit 21 acquires various data from the aircraft 10 via the communication network 2 (for example, various behavior data such as the measured position and attitude of the aircraft 10 and imaging data captured by the aircraft 10). Also, various data (for example, control data for controlling the aircraft 10) are acquired from the control device 30 via the communication network 2. Further, the communication unit 21 provides various data (for example, control data for controlling the flying object 10) to the flying object 10 via the communication network 2, and provides the control device 30 with various data via the communication network 2. It provides various data (imaging data captured by the flying object 10).

The storage unit 22 stores size information relating to the size of the image of the subject displayed on the control device 30 when the aircraft 10 captures an image of the subject from a position a predetermined distance away. Here, conditions such as the distance between the imaging device of the aircraft 10 and the subject at the time of imaging, the angle of view and magnification of the imaging device at that time, and the display magnification of the control device 30 are determined in advance. On the other hand, the size of the display area of the display device of the control device 30 is also fixed. Therefore, when the flying object 10 captures an image of the subject under predetermined conditions, the size of the subject displayed on the display device of the control device 30 (for example, the size of the image of the power transmission line displayed on the display device of the control device 30) thickness) is also uniquely determined. The storage unit 22 stores information indicating the size of the image of the subject displayed on the control device 30 when the aircraft 10 captures an image of the subject from a position a predetermined distance away (for example, the display device of the control device 30). and the size of the display area of the display device of the control device 30).

The control unit 23 causes the display device of the control device 30 to display the image captured by the flying object 10, and causes the control device 30 to display a guide image having a size corresponding to the size information stored in the storage unit 22. Controls display on the device.

Here, an example of the guidance image will be described with reference to FIG. In the example of FIG. 7 , an image C of a power transmission line, which is a subject, is displayed in the display area D of the display device of the control device 30 . As described above, the relationship between the thickness of the power transmission line image displayed on the display device of the control device 30 and the size of the display area of the display device of the control device 30 is predetermined. The interval between the dashed guide images A1 and A2 means the thickness of the transmission line determined by the relationship. In addition, an image corresponding to the power transmission line can be extracted from the captured image by a well-known image analysis method, and a solid-line guidance image A3 is displayed at a position corresponding to the center of the power transmission line in the width direction. ing. That is, the control unit 23 performs control such that the guide images A1 and A2 are displayed at positions separated by a predetermined distance from the guide image A3. In this case, the control unit 23 recognizes the longitudinal direction of the subject (that is, the direction in which the power lines extend) by image analysis, and makes the longitudinal direction parallel to the extending directions of the guide images A1, A2, and A3.

The operator adjusts the distance between the flying object 10 and the subject power transmission line so that the image C of the power transmission line displayed on the display device of the control device 30 is just between the guide images A1 and A2. maneuver. In other words, when the width of the displayed power transmission line image C is wider than the interval between the guide images A1 and A2, the operator may steer the flying object 10 away from the subject. In the case where the width of the image C of the power transmission line is narrower than the interval between the guide images A1 and A2, the aircraft 10 may be steered so as to approach the subject.

[motion]
Next, operation of the server device 20 will be described. In the following description, when the server device 20 is described as the main body of processing, specifically, the processor 2001 and the memory 2002 are loaded with predetermined software (program) on hardware such as the processor 2001 and the memory 2002. performs an operation and controls communication by the communication device 2004 and data reading and/or writing in the memory 2002 and storage 2003 to execute processing. The same applies to the flying object 10 as well.

In FIG. 6, the communication unit 21 of the server device 20 acquires imaging data via the communication network 2 from the aircraft 10 that has started imaging the subject (step S11).

Next, the control unit 23 generates a guide image based on the size information and the imaging data stored in the storage unit 22, specifies the position, size, and direction of the guide image (step S12). is superimposed on the captured image and transmitted to the control device 30 (step S13). The control device 30 causes the display device to display an image as illustrated in FIG. Such processing is repeated until imaging by the flying object 10 ends (step S14; YES).

According to the first embodiment described above, it is possible to focus on a subject and take an image without preparing, for example, a distance image for each subject as in the conventional art.

[Second embodiment]
[composition]
The flying object 10 may be a flying object that flies according to the operation of the control device 30 by the operator (so-called manually controlled flight) as in the first embodiment, or may fly under the control of the server device ( It may be a flying object that performs so-called autopilot flight). In the second embodiment, an example of performing the latter autopilot flight will be described. FIG. 10 is a diagram showing an example of the configuration of a flight system 1a according to the second embodiment. The flight system 1a includes an unmanned flying object 10a called a drone, a server device 20a, a display device 40 for displaying an image captured by the flying object 10a, and a communication network 2 that communicably connects them. . The server device 20a functions as a control device that controls the flight of the aircraft 10a. The inspector refers to the image displayed on the display device 40 connected to the communication network 2 and inspects whether or not the power transmission line, which is the subject, is flawless.

The flying object 10a is equipped with a positioning device that measures its position using GPS, and the flight of the flying object 10a is controlled by the server device 20a based on this positioning position. However, since the position of an object such as a power line cannot be specified accurately, for example, on the order of several centimeters, even if the flight of the aircraft 10a is controlled as planned based on the positioning position by GPS, the aircraft 10a The distance between the lens and the object cannot be maintained accurately enough to satisfy the focus accuracy required at the time of imaging.

FIG. 11 is a diagram showing an example of the functional configuration of the flight system 1a according to the second embodiment. In the server device 20a, the communication unit 21a acquires various data from the flying object 10a via the communication network 2 (for example, various behavior data such as the measured position and attitude of the flying object 10a, and image data captured by the flying object 10a). do. The communication unit 21a also provides various data (for example, control data for controlling the flying object 10a) to the flying object 10a via the communication network 2, and provides the display device 40 with various data via the communication network 2. , provides imaging data imaged by the flying object 10a.

The storage unit 22a stores size information relating to the size of the image of the subject displayed on the display device 40 when the flying object 10a captures an image of the subject from a position a predetermined distance away. Here, as in the first embodiment, conditions such as the distance between the imaging device of the flying object 10a and the subject at the time of imaging, the angle of view and magnification of the imaging device at that time, and the display magnification of the display device 40. is determined in advance, and the size of the display area of the display device 40 is also determined. Therefore, the size of the subject displayed on the display device 40 when the flying object 10a captures an image of the subject under predetermined conditions is also determined. The storage unit 22a stores information indicating the size of the image of the subject displayed on the display device 40 when the aircraft 10a captures an image of the subject from a position separated by a predetermined distance (for example, the size of the image displayed on the display device 40). and the size of the display area of the display device 40).

The control unit 23a is a control unit that controls the flight of the aircraft 10a, and the image of the subject captured by the aircraft 10a is displayed on the display device 40 in a size corresponding to the stored size information. The flying object 10a is made to fly at a position where 7, the controller 23a controls the flying object 10a and the flying object 10a so that the image of the power transmission line displayed on the display device 40 has a size corresponding to the stored size information. Control is performed to adjust the distance from the power transmission line, which is the subject. Specifically, when the width of the displayed power line image is larger than the size corresponding to the stored size information, the control unit 23a causes the flying object 10a to fly away from the subject. Control data instructing to do so is transmitted to the flying object 10a for control. On the other hand, when the width of the displayed power line image is narrower than the size corresponding to the stored size information, the control unit 23a instructs the flying object 10a to fly closer to the subject. control data to control the flying object 10a.

[motion]
Next, the operation of the server device 20a will be described. In the following description, when the server device 20a is described as the subject of processing, specifically, by causing the processor 2001 and the memory 2002 to read predetermined software (program) onto hardware such as the processor 2001 and the memory 2002, the processor 2001 performs an operation and controls communication by the communication device 2004 and data reading and/or writing in the memory 2002 and storage 2003 to execute processing. The same applies to the flying object 10a.

In FIG. 12, the communication unit 21a of the server device 20a acquires imaging data via the communication network 2 from the aircraft 10a that has started imaging (step S21).

Next, the control unit 23a generates control data for controlling the flight of the aircraft 10a based on the size information and the imaging data stored in the storage unit 22a, and transmits the control data to the aircraft 10a to control the flight. control (step S22). Furthermore, the control unit 23a transmits the imaging data acquired from the flying object 10a to the display device 40 (step S23). Such processing is repeated until imaging by the flying object 10a ends (step S24; YES).

According to the second embodiment described above, it is possible to focus on a subject and take an image without preparing, for example, a distance image for each subject as in the conventional art. Furthermore, the operator does not have to manually operate the aircraft 10a.

The invention is not limited to the embodiments described above. The embodiment described above may be modified as follows. Also, two or more of the following modified examples may be combined for implementation.
[Modification 1]
In the first embodiment, the control unit 23 enlarges a partial area of an image captured by the aircraft 10 by a certain magnification and causes the control device 30 to display the image displayed by the control device 30. When the image of the subject is no longer included in the image of the subject, the image captured by the aircraft 10 may be displayed on the control device 30 at a reduced magnification.

This will be described with reference to FIGS. In FIG. 8, the control unit 23 enlarges a part of the imaging range P captured by the flying object 10 at a certain timing (for example, the display range d1 corresponding to the center of the imaging range P). , for example, is displayed on the operating device 30 as illustrated in FIG. Since the position of the flying object 10 can fluctuate due to external factors such as wind, the position of the imaging range of the flying object 10 changes due to the fluctuation, and accordingly the display range of the control device 30 changes from the display range d1 to the display range d2. Suppose it happened. In this case, the control unit 23 can determine that the subject image C is no longer included in the display range d2 by means of image analysis or the like. In such a case, the control unit 23 causes the control device 30 to display an image included in a display range d3 obtained by reducing the magnification of the display range d2. That is, the control unit 23 transmits the guide image to the control device 30 in a state in which the guide image is superimposed on the imaging data representing the image included in the display range d3 obtained by reducing the enlargement magnification of the range corresponding to the center of the imaging range P, and displays it. Let By doing so, the operator does not lose sight of the subject on the control device 30 .

[Modification 2]
In Modification 1 above, the control unit 23 causes the control device 30 to display an image captured by the flying object 10 at a reduced magnification, and then displays a partial area of the image captured by the flying object 10 . When the image of the subject comes to be included in the image of the subject, the image may be displayed on the control device 30 by increasing the enlargement magnification. That is, when the control unit 23 determines that the image C of the subject is included in a predetermined central range of the display range d3 with the reduced magnification by a method such as image analysis, the control unit 23 restores the magnification ( that is, to increase it). In this case, the control unit 23 counts the number of times the enlargement magnification is changed per unit time as described above, and displays the image captured by the flying object 10 on the control device 30 at the enlargement magnification corresponding to the number of times of change. You can let it run. For example, if the enlargement magnification is frequently changed as described above, it may become very difficult for the operator to see the image of the subject, and it may become difficult to perform appropriate maneuvering. In order to avoid such a situation, when the number of times the magnification rate is changed per unit time exceeds the threshold, the magnification rate of the display range is set higher than when the number of times the magnification rate is changed per unit time is equal to or less than the threshold. to be smaller. That is, a wider range of the imaging range is displayed. By doing so, the number of times the enlargement magnification is changed per unit time is reduced as a result, and the difficulty in viewing the image for the operator is alleviated, making it easier to operate.

[Modification 3]
In the second embodiment, the control unit 23a magnifies a partial area of the image captured by the aircraft 10a by a certain magnification and displays it on the display device 40. , the flight of the flying object 10a or the direction of imaging by the flying object 10a may be controlled. As described in the second embodiment, even if the flight of the aircraft 10a is controlled so that the size of the subject is constant, the image of the subject in the display area may be affected by external factors such as wind as described above. If the position is constantly changing, it may be difficult for the inspector to confirm the image. Therefore, the control unit 23a controls the flight of the flying object 10a or the direction of imaging of the flying object 10a by the imaging device so that the subject falls within a predetermined range corresponding to, for example, the center of the display range. Specifically, when the subject image moves in one direction (for example, rightward) in the display range and disappears from the display range, the control unit 23a Flying to the flying object 10a so that the space in the direction (for example, rightward) is included in the imaging range, or the space in the above one direction (for example, rightward) as viewed from the displayed space is in the imaging range. Control data is transmitted to the flying object 10a to instruct the flying object 10a to move the imaging direction of the imaging device of the flying object 10a so as to be included.

[Modification 4]
In the second embodiment, the server device 20 is provided with a condition storage unit that stores conditions that make it difficult for an inspector, who is a viewer of the image, to identify the image of the subject when the image of the subject is captured. may control the flying object 10a so as not to meet the conditions stored in the condition storage unit. This condition relates to the inclusion of an image of a structure other than the object in the background of the image of the object when the object is imaged. This is a condition that the imaging position and imaging direction are such that a structure such as a steel tower is included in the captured image with respect to a subject such as a power transmission line. Further, the conditions may be related to the relationship between the direction of the sunlight when the subject is imaged and the imaging direction of the flying object 10a. This is, for example, a condition under which sunlight does not become the subject's front light. These conditions can be predicted based on the position of the subject, the scheduled flight path of the flying object 10a, the imaging direction of the object by the flying object 10a, the predicted position of the sun, the date and time of imaging, and the like. The condition is stored in the condition storage unit.

[Modification 5]
As in the first embodiment, the flying object 10 performs so-called manual-pilot flight according to the operation of the control device 30 by the operator, and so-called auto-pilot flight under the control of the server device as in the second embodiment. It may be provided with both of the functions to In this case, the flight system includes both the control device 30 and the display device 40 described in the first and second embodiments. Then, when the flying object 10 performs a manually controlled flight, the control unit of the server device causes the control device 30 to display the image captured by the flying object 10, and also displays the image captured by the flying object 10 in a size corresponding to the stored size information. Control is performed to display a guide image of size on the control device 30, and when the flying object 10 performs autopilot flight, the image of the subject captured by the flying object 10 is displayed according to the stored size information. Control is performed to fly the aircraft 10 to a position displayed on the display device 40 in terms of size. In other words, the control unit switches these controls according to the control mode of the aircraft 10 .

[Modification 6]
The control device according to the present invention may be mounted in a server device as in the above embodiment, may be mounted in an aircraft, may be mounted in a control device or a display device, It may be implemented in other devices.

[Other Modifications]
It should be noted that the block diagrams used in the description of the above embodiments show blocks in units of functions. These functional blocks (components) are implemented by any combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be implemented using one device physically or logically coupled, or directly or indirectly using two or more physically or logically separated devices (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices. A functional block may be implemented by combining software in the one device or the plurality of devices.

Functions include judging, determining, determining, calculating, calculating, processing, deriving, examining, searching, checking, receiving, transmitting, outputting, accessing, resolving, selecting, choosing, establishing, comparing, assuming, expecting, assuming, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc. can't For example, a functional block (component) that makes transmission work is called a transmitting control unit (transmitting unit) or a transmitter (transmitter). In either case, as described above, the implementation method is not particularly limited.

For example, a server device or the like according to an embodiment of the present disclosure may function as a computer that performs processing of the present disclosure.

Each aspect/embodiment described in the present disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark) )), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, other suitable systems, and extended It may be applied to at least one of the next generation systems. Also, a plurality of systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A and 5G, etc.).

The processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in this disclosure may be rearranged as long as there is no contradiction. For example, the methods described in this disclosure present elements of the various steps using a sample order, and are not limited to the specific order presented.

Input/output information and the like may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

The determination may be made by a value represented by one bit (0 or 1), by a true/false value (Boolean: true or false), or by numerical comparison (for example, a predetermined value).

Although the present disclosure has been described in detail above, it should be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described in this disclosure. The present disclosure can be practiced with modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Accordingly, the description of the present disclosure is for illustrative purposes and is not meant to be limiting in any way.

Software, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, includes instructions, instruction sets, code, code segments, program code, programs, subprograms, and software modules. , applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like. Software, instructions, information, etc. may also be sent and received over a transmission medium. For example, the software uses wired technology (coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), etc.) and/or wireless technology (infrared, microwave, etc.) to create websites, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.

Information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of
The terms explained in this disclosure and the terms necessary for understanding the present disclosure may be replaced with terms having the same or similar meanings.

In addition, the information, parameters, etc. described in the present disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using other corresponding information. may be represented.

As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Any reference to elements using the "first," "second," etc. designations used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, reference to a first and second element does not imply that only two elements can be employed or that the first element must precede the second element in any way.

The "unit" in the configuration of each device described above may be replaced with "means", "circuit", "device", or the like.

Where "include," "including," and variations thereof are used in this disclosure, these terms are inclusive, as is the term "comprising." is intended. Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive OR.

In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure may include the plural nouns following these articles.

In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean that "A and B are different from C". Terms such as "separate," "coupled," etc. may also be interpreted in the same manner as "different."

1, 1a: flight system, 2: communication network, 10: aircraft, 1001: processor, 1002: memory, 1003: storage, 1004: communication device, 1005: input device, 1006: output device, 1007: flight device, 1008 : sensor 1009: positioning device 20, 20a: server device 21, 21a: communication unit 22, 22a: storage unit 23, 23a: control unit 2001: processor 2002: memory 2003: storage 2004: communication device, 2005: input device, 2006: output device, 30: control device, 3001: processor, 3002: memory, 3003: storage, 3004: communication device, 3005: input device, 3006: output device, 40: display device, C: image of power transmission line, A1, A2, A3: guidance image, D: display area, P: imaging range, d1, d2, d3: display range.

Claims (3)

a storage unit for storing size information relating to the size of an image of a subject displayed on a control device for controlling the flying vehicle when the flying vehicle captures an image of the subject from a position separated by a predetermined distance;
a control unit that causes the control device to display an image captured by the flying object, and causes the control device to display a guide image having a size corresponding to the stored size information ,
The control unit
When a partial area of an image captured by the aircraft is enlarged by a certain magnification and displayed on the control device, and the image displayed by the control device no longer includes the image of the subject. displaying the image picked up by the flying object on the control device at a reduced magnification;
The control unit reduces the magnification of the image captured by the flying object and causes the control device to display the image, and then displays the image of the subject in a partial area of the image captured by the flying object. is included, when displaying on the control device with the enlargement magnification increased, an image captured by the flying object is obtained at the enlargement magnification corresponding to the number of times the enlargement magnification is changed per unit time. display an image on the control device
A control device characterized by: a storage unit for storing size information relating to the size of an image of a subject to be displayed on a display device when the flying object captures an image of the subject from a position separated by a predetermined distance;
a control unit for causing the flying object to fly to a position where the image of the subject imaged by the flying object is displayed on the display device in a size corresponding to the stored size information;
Regarding a condition in which it is difficult for a viewer to identify an image of the subject when the subject is captured, and an image of a structure other than the subject is included in the background of the image of the subject when the subject is captured a condition storage unit that stores conditions;
with
The control unit
When a partial area of an image captured by the flying object is enlarged by a certain magnification and displayed on the display device, the flying object is arranged so that the subject fits within the partial area of the image. Controlling the direction of flight or imaging by the flying object,
Further, the flying object is controlled so as not to meet the conditions stored in the storage unit.
A control device characterized by : The control unit causes the control device to display an image captured by the flying object, and controls the control device to display a guide image having a size corresponding to the stored size information; switching control to fly the aircraft to a position where the image of the subject captured by the body is displayed on a display device in a size corresponding to the stored size information; The control device according to claim 1 or 2 .

Images