WO2023276454A1 - 制御装置、基地局、制御方法、及びプログラム - Google Patents

制御装置、基地局、制御方法、及びプログラム Download PDF

Info

Publication number
WO2023276454A1
WO2023276454A1 PCT/JP2022/019851 JP2022019851W WO2023276454A1 WO 2023276454 A1 WO2023276454 A1 WO 2023276454A1 JP 2022019851 W JP2022019851 W JP 2022019851W WO 2023276454 A1 WO2023276454 A1 WO 2023276454A1
Authority
WO
WIPO (PCT)
Prior art keywords
imaging
flying object
distance
ranging
flight
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/019851
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
哲 和田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023531486A priority Critical patent/JPWO2023276454A1/ja
Publication of WO2023276454A1 publication Critical patent/WO2023276454A1/ja
Priority to US18/534,713 priority patent/US20240111311A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C13/00—Control systems or transmitting systems for actuating flying-control surfaces, lift-increasing flaps, air brakes, or spoilers
    • B64C13/02—Initiating means
    • B64C13/16—Initiating means actuated automatically, e.g. responsive to gust detectors
    • B64C13/18—Initiating means actuated automatically, e.g. responsive to gust detectors using automatic pilot
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C27/00—Rotorcraft; Rotors peculiar thereto
    • B64C27/04—Helicopters
    • B64C27/08—Helicopters with two or more rotors
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00—Equipment not otherwise provided for
    • B64D47/08—Arrangements of cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U20/00—Constructional aspects of UAVs
    • B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
    • B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U30/00—Means for producing lift; Empennages; Arrangements thereof
    • B64U30/20—Rotors; Rotor supports
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U50/00—Propulsion; Power supply
    • B64U50/10—Propulsion
    • B64U50/19—Propulsion using electrically powered motors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/22—Command input arrangements
    • G05D1/221—Remote-control arrangements
    • G05D1/225—Remote-control arrangements operated by off-board computers
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/242—Means based on the reflection of waves generated by the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/247—Arrangements for determining position or orientation using signals provided by artificial sources external to the vehicle, e.g. navigation beacons
    • G05D1/249—Arrangements for determining position or orientation using signals provided by artificial sources external to the vehicle, e.g. navigation beacons from positioning sensors located off-board the vehicle, e.g. from cameras
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/60—Intended control result
    • G05D1/656—Interaction with payloads or external entities
    • G05D1/689—Pointing payloads towards fixed or moving targets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/25—UAVs specially adapted for particular uses or applications for manufacturing or servicing
    • B64U2101/26—UAVs specially adapted for particular uses or applications for manufacturing or servicing for manufacturing, inspections or repairs
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2105/00—Specific applications of the controlled vehicles
    • G05D2105/80—Specific applications of the controlled vehicles for information gathering, e.g. for academic research
    • G05D2105/89—Specific applications of the controlled vehicles for information gathering, e.g. for academic research for inspecting structures, e.g. wind mills, bridges, buildings or vehicles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2109/00—Types of controlled vehicles
    • G05D2109/20—Aircraft, e.g. drones
    • G05D2109/25—Rotorcrafts
    • G05D2109/254—Flying platforms, e.g. multicopters
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/10—Optical signals

Definitions

  • the technology of the present disclosure relates to control devices, base stations, control methods, and programs.
  • Japanese Patent Application Laid-Open No. 2017-151008 discloses that a retroreflector of a flying object equipped with a retroreflector is irradiated with tracking light, the tracking light is received, and optical tracking for tracking the flying object based on the light reception result, and flight an image of the body is obtained, the flying object is detected from the image, and the flying object is tracked based on the detection result; There is disclosed a flying object tracking method for returning to optical tracking based on the detection result of image tracking when tracking becomes impossible.
  • Japanese Patent Application Laid-Open No. 2014-104797 describes a moving mechanism that moves on the floor and enters the building, a camera provided in the moving mechanism, a pan-tilt mechanism of the camera, a flying object that can be mounted on the moving mechanism, A luminous body provided on the flying body, a pan-tilt control means for controlling the pan-tilt mechanism so that the camera tracks the luminous body, a display means for displaying an image taken by the camera, and an operating means for operating at least the flying body. and an in-building investigation system is disclosed.
  • Japanese Unexamined Patent Application Publication No. 2018-173960 discloses an information processing system that performs flight control of an unmanned airplane, and when the unmanned airplane is flying in a position that is photographed by a network camera, it is possible to fly in a position that is not photographed by the network camera.
  • An information processing system is disclosed comprising control means for controlling the flight of an unmanned aerial vehicle to do so.
  • Japanese Patent Application Laid-Open No. 2018-070013 discloses an unmanned aircraft control system in which an unmanned aircraft connected to a base station by a cable and an information processing device are connected via a network. and cable adjustment means for controlling the length of the cable to be shorter than the width of the base station when the length of the cable is determined to be longer than the width of the base station by the comparison means.
  • An unmanned aerial vehicle control system is disclosed.
  • a moving body identification system for identifying a moving body, in which first position information of a plurality of moving bodies detected by a moving state monitoring device that monitors the moving state of the moving body is obtaining movement state information including the moving state information, obtaining predetermined report information including second position information of the mobile body measured by the mobile body, based on the first position information and the second position information, A mobile identification system is disclosed for identifying the registration status of a mobile.
  • the object is imaged by a first imaging device mounted on a flying object that flies along the object.
  • a control device for example, a control device, a base station, a control method, and a program capable of maintaining a constant resolution of an image obtained by the above.
  • a first aspect of the technology of the present disclosure includes a processor and a memory connected to or built into the processor, wherein the processor rotates the range finder with respect to a rotary drive device to which the range finder is attached. causing the ranging device to measure a first distance between the object and the ranging device at a plurality of ranging points on the object; based on the first distance measured for each ranging point, setting a flight route along which the flying object flies along the object, causing the flying object to fly along the flight route, and capturing a plurality of subjects of the object with respect to a first imaging device mounted on the flying object; The control device performs control to keep the pixel resolution of the first imaging device constant when acquiring a plurality of first images by imaging an imaging region.
  • a second aspect of the technology of the present disclosure is the control device according to the first aspect, wherein the processor sets the rotation angle of the rotary drive device to a second rotation angle at which the flying object is included in the ranging range of the ranging device. and causing the rangefinder to measure a second distance between the aircraft and the rangefinder, and flying the flight route to the aircraft based on the second rotation angle and the second distance It is a control device that controls the
  • a third aspect of the technology of the present disclosure is the control device according to the second aspect, wherein the ranging device includes a LiDAR scanner, the second distance is a distance between the flying object and the LiDAR scanner, The processor derives a second absolute coordinate of the flying object based on the first absolute coordinate of the rotary drive device, the second rotation angle, the angle of the laser beam irradiated from the LiDAR scanner toward the flying object, and the second distance. and controls the aircraft to fly a flight route based on the second absolute coordinates.
  • a fourth aspect of the technology of the present disclosure is the control device according to the second aspect or the third aspect, wherein the rotary drive device is attached with a second imaging device, and the processor comprises the second imaging device is a control device that performs control to adjust the rotation angle of the rotary drive device to a second rotation angle based on a second image obtained by capturing an image of the flying object by.
  • a fifth aspect of the technology of the present disclosure is the control device according to the fourth aspect, wherein the second rotation angle is an angle at which the flying object is positioned at the center of the angle of view of the second imaging device. be.
  • a sixth aspect of the technology of the present disclosure is the control device according to the fourth aspect or the fifth aspect, wherein the aircraft includes a plurality of members classified in different manners, and the processor converts the second image to A control device for controlling the attitude of an aircraft based on the positions of a plurality of photographed members.
  • a seventh aspect according to the technology of the present disclosure is the control device according to the sixth aspect, wherein the different aspects are different colors and the member is a propeller.
  • An eighth aspect of the technology of the present disclosure is the control device according to the sixth aspect, wherein the different aspects are different colors and the member is a light emitter.
  • a ninth aspect of the technology of the present disclosure is the control device according to the sixth aspect, wherein the different aspect is a different blinking pattern, and the member is a light emitter.
  • a tenth aspect of the technology of the present disclosure is the control device according to any one of the first to ninth aspects, wherein the plurality of first images is a plurality of images in which the flying object is set on the flight route. is an image acquired each time it reaches each of the first imaging positions of .
  • An eleventh aspect of the technology of the present disclosure is the control device according to the tenth aspect, wherein the plurality of first imaging positions are first imaging positions acquired at adjacent first imaging positions among the plurality of first imaging positions. A control device where the images partially overlap.
  • a twelfth aspect of the technology of the present disclosure is the control device according to any one of the first to eleventh aspects, wherein the surface of the object has a concave portion, and
  • the processor is a controller that sets a flight route on a smooth virtual surface facing the surface when the area of the opening of the recess is smaller than the predetermined area.
  • a thirteenth aspect of the technology of the present disclosure is the control device according to the twelfth aspect, wherein the processor, when the flying object flies over the concave portion, selects one of the zoom lens and the focus lens of the first imaging device. It is a control device that performs control to keep the pixel resolution constant by operating at least one of them.
  • a fourteenth aspect of the technology of the present disclosure is the control device according to any one of the first to thirteenth aspects, wherein the processor is attached with a first rangefinder as a rangefinder.
  • a first distance measuring device is rotated with respect to a first rotation driving device as a rotation driving device, and a first distance is obtained at a plurality of first distance measuring locations among a plurality of distance measuring locations with respect to the first ranging device.
  • the second range finder is rotated with respect to the second rotary drive device as the rotary drive device to which the second range finder as the range finder is attached, and with respect to the second range finder, A first distance is measured at a plurality of second ranging points among the plurality of ranging points, and the first distance is measured at each first ranging point and the first distance is measured at each second ranging point.
  • a fifteenth aspect of the technology of the present disclosure is the control device according to the fourteenth aspect, wherein the processor measures the first distance measured by the second rangefinder based on predetermined first calibration information, A control device for converting a distance based on the position of the first rangefinder.
  • a sixteenth aspect of the technology of the present disclosure is the control device according to the fourteenth aspect or the fifteenth aspect, wherein the processor, based on predetermined second calibration information, measures A control device that converts the position of the flying object into a position based on the position of the first rangefinder.
  • a seventeenth aspect of the technology of the present disclosure is the control device according to any one of the fourteenth to sixteenth aspects, wherein the processor, in accordance with the position of the flying object, A control device for selecting a rangefinder for measuring the position of an aircraft from among the second rangefinders.
  • An eighteenth aspect of the technology of the present disclosure is the control device according to any one of the fourteenth to seventeenth aspects, wherein the processor comprises: When a flight route is set based on a point located outside the second ranging area of the ranging device, the distance between the point and the first ranging device is calculated from the position of the point with respect to the first ranging device. and a control device for deriving based on the angle of the direction and the distance between the first rangefinder and the second rangefinder.
  • a nineteenth aspect of the technology of the present disclosure is the control device according to the eighteenth aspect, wherein when the flying object is positioned outside the first ranging area and the second ranging area, the processor Control for deriving the distance between the first rangefinder based on the angle of the direction in which the aircraft is positioned with respect to the first rangefinder and the distance between the first rangefinder and the second rangefinder It is a device.
  • a twentieth aspect of the technology of the present disclosure is the control device according to any one of the first to nineteenth aspects, wherein the flying object includes a third imaging device, Based on the third image obtained by imaging the object with the third imaging device when the flying object that has moved from the set second imaging position reaches the third imaging position set on the flight route A position correction process for correcting the position of the flying object is performed, and the position correcting process causes the third imaging device to pick up an image of the object when the flying object reaches the second imaging position, thereby obtaining a fourth image.
  • the fourth image and the third image are acquired. This is processing for correcting the position of the flying object to a position where the amount of overlap between the fourth image and the fifth image becomes the predetermined amount of overlap based on the overlap amount of .
  • a twenty-first aspect of the technology of the present disclosure is a base station including a control device according to any one of the first to twentieth aspects, a rotary drive device, and a distance measuring device.
  • a twenty-second aspect of the technology of the present disclosure is to rotate the distance measuring device with respect to a rotary drive device to which the distance measuring device is attached, and rotate the distance measuring device with respect to a plurality of distance measuring points of the object.
  • measuring a first distance between the object and the rangefinder setting a flight route for the aircraft to fly along the object based on the first distance measured at each range-finding location; and acquiring a plurality of first images by causing the aircraft to fly along a flight route and causing a first image pickup device mounted on the aircraft to image a plurality of areas to be imaged of the object.
  • the control method includes performing control to keep the pixel resolution of the first imaging device constant when the first imaging device is to be set.
  • a twenty-third aspect of the technology of the present disclosure is to rotate the distance measuring device with respect to the rotation drive device to which the distance measuring device is attached, measuring a first distance between the object and the rangefinder; setting a flight route for the aircraft to fly along the object based on the first distance measured at each range-finding location; and acquiring a plurality of first images by causing the aircraft to fly along a flight route and causing a first image pickup device mounted on the aircraft to image a plurality of areas to be imaged of the object.
  • a program for causing a computer to execute processing including performing control to keep the pixel resolution of the first imaging device constant when the image is to be captured.
  • FIG. 1 is a side view showing an example of an inspection system according to a first embodiment of technology of the present disclosure
  • FIG. 1 is a plan view showing an example of an inspection system according to a first embodiment of technology of the present disclosure
  • FIG. 1 is a plan view showing an example of an aircraft according to a first embodiment of the technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of a base station according to a first embodiment of technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of a base station rotary drive device according to a first embodiment of the technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of an imaging device of a base station according to the first embodiment of the technique of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of a distance measuring device of a base station according to the first embodiment of the technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of an aircraft according to a first embodiment of the technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of an imaging device for a flying object according to a first embodiment of the technology of the present disclosure
  • FIG. FIG. 4 is a block diagram showing an example of a functional configuration of a processor of the base station according to the first embodiment of the technology of the present disclosure
  • It is a block diagram showing an example of functional composition of a flight route setting processing part concerning a 1st embodiment of art of this indication.
  • FIG. 1 is a block diagram showing an example of an electrical configuration of a distance measuring device of a base station according to the first embodiment of the technology of the present disclosure
  • FIG. 1 is a block diagram showing an example of an electrical configuration of an aircraft according to a first embodiment of the technology
  • FIG. 3 is a block diagram showing an example of a functional configuration of a flight control processing unit according to the first embodiment of the technology of the present disclosure
  • 2 is a block diagram showing an example of a functional configuration of an imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 3 is a block diagram showing an example of a functional configuration of a processor of the aircraft according to the first embodiment of the technology of the present disclosure
  • FIG. 5 is an explanatory diagram illustrating an example of the first operation of the flight route setting processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 5 is an explanatory diagram illustrating an example of a second operation of the flight route setting processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a third operation of the flight route setting processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a fourth operation of the flight route setting processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a fifth operation of the flight route setting processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 4 is an explanatory diagram illustrating an example of a first operation of the flight control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 5 is an explanatory diagram illustrating an example of a second operation of the flight control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a third operation of the flight control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 5 is an explanatory diagram illustrating an example of a first operation of the imaging control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 7 is an explanatory diagram illustrating an example of a second operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 7 is an explanatory diagram illustrating an example of a third operation of the imaging control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a fourth operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 5 is an explanatory diagram illustrating an example of a first operation of the imaging control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 7 is an explanatory diagram illustrating an example of a second operation of the imaging control processing unit according to
  • FIG. 11 is an explanatory diagram illustrating an example of a fifth operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of a sixth operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of a seventh operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of an eighth operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 20 is an explanatory diagram illustrating an example of a ninth operation of the imaging control processing unit according to the first embodiment of the technology of the present disclosure
  • FIG. 20 is an explanatory diagram illustrating an example of a tenth operation of the imaging control processing unit according to the first embodiment of the technique of the present disclosure
  • FIG. 20 is an explanatory diagram illustrating an example of an eleventh operation of the imaging control processing unit according to the first embodiment of the technology of the present disclosure
  • 6 is a flow chart showing an example of the flow of the first process of the flight imaging support process according to the first embodiment of the technology of the present disclosure
  • 7 is a flowchart showing an example of the flow of second processing of flight imaging support processing according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is a flowchart showing an example of the flow of third processing of flight imaging support processing according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is a flowchart showing an example of a flow of fourth processing of flight imaging support processing according to the first embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flowchart showing an example of the flow of a fifth process of the flight imaging support process according to the first embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flow chart showing an example of the flow of sixth processing of flight imaging support processing according to the first embodiment of the technology of the present disclosure
  • FIG. 4 is a flow chart showing an example of the flow of the first processing of flight imaging processing according to the first embodiment of the technology of the present disclosure
  • 7 is a flowchart showing an example of the flow of second processing of flight imaging processing according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is a flowchart showing an example of the flow of third processing of flight imaging processing according to the first embodiment of the technology of the present disclosure
  • FIG. FIG. 4 is a plan view showing a modified example of the flying object according to the first embodiment of the technology of the present disclosure
  • FIG. 11 is a plan view showing an example of an inspection system according to a second embodiment of the technology of the present disclosure
  • FIG. 7 is a block diagram showing an example of a functional configuration of a flight route setting processing unit according to a second embodiment of the technology of the present disclosure
  • FIG. 7 is a block diagram showing an example of a functional configuration of a flight control processing unit according to a second embodiment of the technology of the present disclosure
  • FIG. 7 is a block diagram showing an example of a functional configuration of an imaging control processing unit according to a second embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a first operation of a flight route setting processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of a second operation of the flight route setting processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 11 is a schematic diagram showing an example of a plurality of points in an area where distance measurement areas of each distance measuring device overlap with each other according to the second embodiment of the technology of the present disclosure
  • FIG. 11 is a schematic diagram showing an example of a plurality of points in an area where distance measurement areas of each distance measuring device overlap with each other according to the second embodiment of the technology of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of a third operation of the flight route setting processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 12 is an explanatory diagram illustrating an example of a fourth operation of the flight route setting processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 7 is an explanatory diagram illustrating an example of the operation of a flight control processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 10 is an explanatory diagram illustrating an example of the operation of an imaging control processing unit according to the second embodiment of the technology of the present disclosure
  • FIG. 11 is a flowchart showing an example of the flow of first processing of flight imaging support processing according to the second embodiment of the technology of the present disclosure
  • FIG. 11 is a flowchart showing an example of a flow of second processing of flight imaging support processing according to the second embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flowchart showing an example of the flow of third processing of flight imaging support processing according to the second embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flow chart showing an example of the flow of a fourth process of flight imaging support processing according to the second embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flowchart showing an example of the flow of a fifth process of flight imaging support processing according to the second embodiment of the technology of the present disclosure
  • FIG. 11 is a block diagram showing an example of a functional configuration of a processor of a base station according to a third embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a first operation of a distance derivation processing unit according to a third embodiment of the technology of the present disclosure
  • FIG. 13 is an explanatory diagram illustrating an example of a second operation of a distance derivation processing unit according to the third embodiment of the technology of the present disclosure
  • FIG. 11 is a schematic diagram showing an example of points located outside the range-finding area of each range-finding device according to the third embodiment of the technology of the present disclosure
  • FIG. 11 is a block diagram showing an example of a functional configuration of a processor of a base station according to a third embodiment of the technology of the present disclosure
  • FIG. 11 is an explanatory diagram illustrating an example of a first operation of a distance derivation processing unit according to a third embodiment of the technology of the present disclosure
  • FIG. 13 is an explan
  • FIG. 11 is an explanatory diagram illustrating an example of distance derivation processing according to the third embodiment of the technology of the present disclosure
  • FIG. 12 is a block diagram showing an example of a functional configuration of a processor of a base station according to the fourth embodiment of the technology of the present disclosure
  • FIG. 12 is a block diagram showing an example of a functional configuration of a position correction processing unit according to the fourth embodiment of the technique of the present disclosure
  • FIG. 14 is a block diagram showing an example of a first operation of a position correction processing unit according to the fourth embodiment of the technique of the present disclosure
  • FIG. 11 is a flow chart showing an example of the flow of the first process of the position correction process according to the fourth embodiment of the technology of the present disclosure
  • FIG. FIG. 11 is a flowchart showing an example of the flow of second processing of position correction processing according to the fourth embodiment of the technology of the present disclosure
  • CPU is an abbreviation for "Central Processing Unit”.
  • GPU is an abbreviation for "Graphics Processing Unit”.
  • RAM is an abbreviation for "Random Access Memory”.
  • NVM is an abbreviation for "Non-volatile memory”.
  • IC is an abbreviation for "Integrated Circuit”.
  • ASIC is an abbreviation for "Application Specific Integrated Circuit”.
  • PLD is an abbreviation for "Programmable Logic Device”.
  • FPGA is an abbreviation for "Field-Programmable Gate Array”.
  • SoC is an abbreviation for "System-on-a-chip.”
  • SSD is an abbreviation for "Solid State Drive”.
  • HDD is an abbreviation for "Hard Disk Drive”.
  • EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory”.
  • SRAM is an abbreviation for "Static Random Access Memory”.
  • I/F is an abbreviation for "Interface”.
  • USB is an abbreviation for "Universal Serial Bus”.
  • CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor”.
  • CCD is an abbreviation for "Charge Coupled Device”.
  • LED is an abbreviation for "light emitting diode”.
  • EL is an abbreviation for "Electro Luminescence”.
  • LiDAR is an abbreviation for “light detection and ranging”.
  • MEMS is an abbreviation for “Micro Electro Mechanical Systems”.
  • AI is an abbreviation for “Artificial Intelligence”.
  • the “horizontal direction” means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs in addition to the complete horizontal direction, and is contrary to the spirit of the technology of the present disclosure. It refers to the horizontal direction in the sense of including the degree of error that does not occur.
  • the “vertical direction” means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the perfect vertical direction, and is contrary to the spirit of the technology of the present disclosure. It refers to the vertical direction in the sense of including the degree of error that does not occur.
  • parallel means, in addition to complete parallelism, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not go against the spirit of the technology of the present disclosure. It refers to parallel in the sense of including the error of
  • symmetry means, in addition to perfect symmetry, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not go against the spirit of the technology of the present disclosure.
  • an inspection system 1 includes an image analysis device 2 and an imaging system S, and inspects an inspection object 3 .
  • the inspection object 3 is a bridge pier.
  • the piers are made of reinforced concrete.
  • Road facilities include, for example, road surfaces, tunnels, guardrails, traffic lights, and/or windbreak fences.
  • the inspection object 3 may be social infrastructure other than road facilities (for example, airport facilities, port facilities, water storage facilities, gas facilities, medical facilities, firefighting facilities, and/or educational facilities, etc.). , may be private property. Also, the inspection object 3 may be land (for example, state-owned land and/or private land).
  • the bridge piers illustrated as the inspection object 3 may be bridge piers other than those made of reinforced concrete.
  • inspection refers to inspection of the state of the inspection object 3, for example.
  • the inspection system 1 inspects whether or not the inspection object 3 is damaged and/or the extent of the damage.
  • the inspection target 3 is an example of the "target" according to the technology of the present disclosure.
  • the imaging system S includes a base station 10 and an aircraft 310.
  • Base station 10 has a control function.
  • the control function is a function of controlling the flying object 310 by giving an instruction such as a flight instruction or an imaging instruction to the flying object 310 .
  • the flying object 310 has a flight function and a first imaging function.
  • the flight function is a function of flying based on flight instructions.
  • the first image capturing function is a function of capturing an image of a subject (in the example shown in FIG. 1, the inspection object 3) based on an image capturing instruction.
  • the flying object 310 will be described in more detail.
  • the communication device 12 is installed in the base station 10 , and the communication device 312 communicates with the communication device 12 .
  • the communication device 312 may communicate with the communication device 12 wirelessly or may communicate with the communication device 12 by wire.
  • a first imaging function is realized by the imaging device 330 .
  • the imaging device 330 may be, for example, a digital camera or a video camera.
  • the imaging device 330 images the second subject (in the example shown in FIG. 1, the inspection target 3).
  • the imaging device 330 is mounted on the upper part of the aircraft body 320, but this is only an example, and the imaging device 330 may be mounted on the lower part of the aircraft body 320.
  • the imaging device 330 is mounted in the central portion of the aircraft main body 320 and is arranged in an orientation to image the front of the aircraft 310 .
  • the imaging device 330 is an example of a “first imaging device” according to the technology of the present disclosure.
  • the imaging system S is a system that provides the image analysis device 2 with image data obtained by imaging the inspection object 3 by the flying object 310 .
  • the image analysis device 2 performs image analysis processing on the image data provided from the imaging system S to inspect the presence or absence of damage and/or the degree of damage to the inspection object 3, and outputs inspection results. do.
  • the image analysis process is a process of analyzing an image using template matching technology and/or artificial intelligence.
  • the base station 10 includes a rotary drive device 20, an imaging device 30, and a distance measuring device 40 in addition to the communication device 12.
  • the rotary drive device 20 has a pedestal 27 .
  • the rotary drive device 20 is a device capable of rotating the pedestal 27 horizontally and vertically. In FIG. 1, arrow V indicates the vertical direction.
  • the imaging device 30 and the distance measuring device 40 are attached to the pedestal 27 .
  • the imaging device 30 is arranged above the distance measuring device 40, but this is only an example, and the imaging device 30 may be arranged below the distance measuring device 40. It may be arranged horizontally along with the distance measuring device 40 .
  • the imaging device 30 is a device having a second imaging function.
  • the second imaging function is a function of imaging an imaging scene including the inspection object 3 or the flying object 310 .
  • the second imaging function is implemented by, for example, a digital camera or a video camera.
  • the imaging device 30 is an example of a “second imaging device” according to the technology of the present disclosure.
  • the distance measuring device 40 is a device having a distance measuring function.
  • the ranging function is a function of measuring the distance between the inspection object 3 or the flying object 310 and the ranging device 40 .
  • the ranging function is implemented by, for example, an ultrasonic ranging device, a laser ranging device, a radar ranging device, or the like.
  • Laser rangefinders include LiDAR scanners. A case in which a LiDAR scanner is used as an example of a laser rangefinder that realizes a rangefinder function will be described below.
  • the direction in which the rangefinder 40 scans with laser light (hereinafter referred to as the scanning direction) is set in the horizontal direction.
  • arrow H indicates the horizontal direction.
  • a distance measuring range 41 which is a range scanned by the distance measuring device 40 with laser light, is set within an imaging range 31 of the imaging device 30 in plan view.
  • a range in which the first subject is located in the center of the distance measurement range 41 when the first subject (for example, the flying object 310 shown in FIGS. 1 and 2) is located in the center of the angle of view of the imaging device 30.
  • the distance measuring range 41 is set.
  • the optical axis OA1 of the imaging device 30 coincides with the central axis AC of the range-finding range 41 in a plan view of the imaging system S. As shown in FIG.
  • the scanning direction of the distance measuring device 40 may be set in the vertical direction, or may be set in both the horizontal direction and the vertical direction.
  • the base station 10 includes the imaging device 30 and the distance measuring device 40, but this is only an example, and the base station 10 has the second imaging function and the measuring function.
  • An imaging device having a distance function may be provided.
  • An imaging device having the second imaging function and the ranging function includes, for example, a stereo camera or a phase difference pixel camera.
  • the aircraft main body 320 is a multicopter having a first propeller 341A, a second propeller 341B, a third propeller 341C, and a fourth propeller 341D.
  • the first propeller 341A is arranged on the front right side of the aircraft body 320
  • the second propeller 341B is arranged on the front left side of the aircraft body 320
  • the third propeller 341C is arranged on the rear side of the aircraft body 320.
  • the fourth propeller 341D is arranged on the rear left side of the aircraft main body 320.
  • the first propeller 341A and the third propeller 341C are arranged on the right side with respect to the imaging device 330, and the second propeller 341B and the fourth propeller 341D are arranged on the left side with respect to the imaging device 330.
  • the first propeller 341A is arranged at a line-symmetrical position with respect to the second propeller 341B centering on the optical axis OA2 of the imaging device 330 in plan view
  • the third propeller 341C is arranged at a line-symmetrical position with respect to the optical axis OA2 of the imaging device 330 in plan view. centered on the fourth propeller 341D.
  • the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D are examples of the "plurality of members" according to the technology of the present disclosure.
  • the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D are classified with different colors as examples of different aspects.
  • dots attached to the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D express the color of each propeller.
  • the color of the first propeller 341A is the same as the color of the second propeller 341B
  • the color of the third propeller 341C is the same as the color of the fourth propeller 341D.
  • the first color set for the first propeller 341A and the second propeller 341B is different from the second color set for the third propeller 341C and the fourth propeller 341D.
  • the first color and the second color may each be chromatic or achromatic.
  • a processor 51 (see FIG. 4) of the base station 10 (to be described later) can distinguish the first color and the second color based on an image obtained by being imaged by the imaging device 30. Any color may be used as long as it is a color.
  • the first color is set for the first propeller 341A and the second propeller 341B
  • the second color is set for the third propeller 341C and the fourth propeller 341D
  • the first color may be set for the first propeller 341A and the third propeller 341C
  • the second color may be set for the second propeller 341B and the fourth propeller 341D
  • a first color may be set for the first propeller 341A and the fourth propeller 341D
  • a second color may be set for the second propeller 341B and the third propeller 341C.
  • different colors may be set for the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D.
  • the base station 10 includes a communication device 12, a reception device 14, a display 16, a rotation drive device 20, an imaging device 30, a distance measurement device 40, and a computer 50.
  • the computer 50 is an example of a "control device” and a “computer” according to the technology of the present disclosure.
  • Computer 50 comprises processor 51 , storage 52 and RAM 53 .
  • the processor 51 is an example of the “processor” according to the technology of the present disclosure
  • the RAM 53 is an example of the “memory” according to the technology of the present disclosure.
  • Processor 51 , storage 52 and RAM 53 are interconnected via bus 54 .
  • Also connected to the bus 54 are the communication device 12 , reception device 14 , display 16 , rotary drive device 20 , imaging device 30 and distance measuring device 40 .
  • one bus is illustrated as the bus 54 for convenience of illustration, but a plurality of buses may be used.
  • Bus 54 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, and the like.
  • the processor 51 has, for example, a CPU and controls the base station 10 as a whole. Although an example in which the processor 51 has a CPU is given here, this is merely an example.
  • processor 51 may have a CPU and a GPU. In this case, for example, the GPU operates under the control of the CPU and is responsible for executing image processing.
  • the storage 52 is a nonvolatile storage device that stores various programs and various parameters. Examples of the storage 52 include HDDs and SSDs. Note that the HDD and SSD are merely examples, and flash memory, magnetoresistive memory, and/or ferroelectric memory may be used in place of or together with the HDD and/or SSD. .
  • the RAM 53 is a memory that temporarily stores information, and is used by the processor 51 as a work memory. Examples of the RAM 53 include DRAM and/or SRAM.
  • the reception device 14 has a keyboard, mouse, touch pad, etc., and receives information provided by the user.
  • the display 16 displays various information (eg, images, characters, etc.) under the control of the processor 51 .
  • Examples of the display 16 include an EL display (such as an organic EL display or an inorganic EL display). Note that the display 16 is not limited to the EL display, and may be another type of display 16 such as a liquid crystal display.
  • the communication device 12 is communicably connected to the aircraft 310 .
  • the communication device 12 is connected so as to be able to communicate wirelessly with the aircraft 310 according to a predetermined wireless communication standard.
  • the predefined wireless communication standard includes, for example, Bluetooth (registered trademark). Note that other wireless communication standards (eg, WI/Fi, 5G, etc.) may be used. Although wireless communication is exemplified here, the technology of the present disclosure is not limited to this, and wired communication may be applied instead of wireless communication.
  • the communication device 12 controls transmission and reception of information with the aircraft 310 . For example, the communication device 12 transmits information requested by the processor 51 to the aircraft 310 .
  • the communication device 12 also receives information transmitted from the aircraft 310 and outputs the received information to the processor 51 via the bus 54 .
  • the rotary drive device 20 includes an input/output I/F 22, a motor driver 23, a pan motor 24, a tilt motor 25, a pan-tilt mechanism 26, and a pedestal 27.
  • the motor driver 23 is connected to the processor 51 via the input/output I/F 22 and bus 54 .
  • the motor driver 23 controls the pan motor 24 and tilt motor 25 according to instructions from the processor 51 .
  • the pan motor 24 and tilt motor 25 are motors such as DC brushed motors, brushless motors, or stepping motors.
  • the pan-tilt mechanism 26 is, for example, a two-axis gimbal, and includes a pan mechanism 28 and a tilt mechanism 29.
  • the pan mechanism 28 is connected to the rotating shaft of the pan motor 24
  • the tilt mechanism 29 is connected to the rotating shaft of the tilt motor 25 .
  • the pedestal 27 is connected to the pan-tilt mechanism 26 .
  • the pan mechanism 28 receives the rotational force of the pan motor 24 to impart a horizontal rotational force to the pedestal 27, and the tilt mechanism 29 receives the rotational force of the tilt motor 25 to impart a vertical rotational force to the pedestal.
  • the pedestal 27 is horizontally rotated by a rotational force applied from the pan motor 24 via the pan mechanism 28 and vertically rotated by a rotational force applied from the tilt motor 25 via the tilt mechanism 29 .
  • the imaging device 30 includes an input/output I/F 32, an image sensor driver 33, and an image sensor .
  • the image sensor driver 33 and image sensor 34 are connected to the processor 51 via the input/output I/F 32 and bus 54 .
  • the image sensor driver 33 controls the image sensor 34 according to instructions from the processor 51 .
  • Image sensor 34 is, for example, a CMOS image sensor. Although a CMOS image sensor is exemplified as the image sensor 34 here, the technology of the present disclosure is not limited to this, and other image sensors may be used.
  • the image sensor 34 captures an image of the first subject (for example, the flying object 310 shown in FIGS. 1 and 2) and outputs the captured image to the processor 51. do.
  • the imaging device 30 includes optical components such as an objective lens, a focus lens, a zoom lens, and an aperture. Also, although not shown, the imaging device 30 includes actuators for driving optical components such as a focus lens, a zoom lens, and a diaphragm. When the imaging device 30 takes an image, optical components such as a focus lens, a zoom lens, and an aperture provided in the imaging device 30 are driven by controlling the actuator.
  • the rangefinder 40 includes an input/output I/F 42, a rangefinder sensor driver 43, a rangefinder sensor 44, a scanner driver 45, and a scanner mechanism 46.
  • the ranging sensor driver 43 , ranging sensor 44 , and scanner driver 45 are connected to the processor 51 via the input/output I/F 42 and bus 54 .
  • the ranging sensor driver 43 controls the ranging sensor 44 according to instructions from the processor 51 .
  • the ranging sensor 44 has a laser light output function, a reflected light detection function, and a distance information output function.
  • the laser light output function is a function to output a laser light
  • the reflected light detection function is a function to detect the reflected light that is the light reflected by an object
  • the distance information output function is a function to detect the laser light. This is a function of outputting distance information (that is, information indicating the distance from the distance measuring sensor 44 to the object) according to the time from output to detection of reflected light.
  • the scanner mechanism 46 is, for example, a galvanomirror scanner or a MEMS mirror scanner, and includes a scanner mirror 47 and a scanner actuator 48.
  • Scanner mirror 47 reflects the laser light.
  • the laser beam reflected by the scanner mirror 47 is applied to an object (for example, the flying vehicle 310 or the inspection object 3 shown in FIG. 1).
  • the scanner actuator 48 changes the angle of the scanner mirror 47 by applying power to the scanner mirror 47 .
  • the reflection angle of the laser beam reflected by the scanner mirror 47 changes in the horizontal direction.
  • an aircraft 310 includes a communication device 312, an image memory 314, an input/output I/F 322, an imaging device 330, a flight device 340, and a computer 350.
  • the computer 350 includes a processor 351, storage 352, and RAM353.
  • the processor 351 , storage 352 and RAM 353 are interconnected via a bus 354 , and the bus 354 is connected to the input/output I/F 322 .
  • a communication device 312 , an image memory 314 , and an imaging device 330 are also connected to the input/output I/F 322 .
  • one bus is shown as the bus 354 for convenience of illustration, but a plurality of buses may be used.
  • Bus 354 may be a serial bus or a parallel bus including a data bus, an address bus, a control bus, and the like.
  • the processor 351 has, for example, a CPU, and controls the aircraft 310 as a whole. Although an example in which the processor 351 has a CPU is given here, this is merely an example.
  • processor 351 may have a CPU and a GPU. In this case, for example, the GPU operates under the control of the CPU and is responsible for executing image processing.
  • the storage 352 is a nonvolatile storage device that stores various programs and various parameters. Examples of the storage 352 include HDDs and SSDs. Note that the HDD and SSD are merely examples, and flash memory, magnetoresistive memory, and/or ferroelectric memory may be used in place of or together with the HDD and/or SSD. .
  • the RAM 353 is a memory that temporarily stores information, and is used by the processor 351 as a work memory. Examples of the RAM 353 include DRAM and/or SRAM.
  • the image memory 314 is, for example, an EEPROM. However, this is merely an example, and an HDD and/or an SSD or the like may be applied as the image memory 314 instead of or together with the EEPROM. Also, the image memory 314 may be a memory card. The image memory 314 stores an image captured by the imaging device 330 .
  • the communication device 312 is communicably connected to the base station 10 .
  • the communication device 312 is in charge of exchanging information with the base station 10 .
  • the communication device 312 transmits information requested by the processor 351 to the base station 10 .
  • Communication device 312 also receives information transmitted from base station 10 and outputs the received information to processor 351 via bus 354 .
  • the flight device 340 has a first propeller 341A, a second propeller 341B, a third propeller 341C, a fourth propeller 341D, a plurality of motors 342, and a motor driver 343.
  • the motor driver 343 is connected to the processor 351 via the input/output I/F 322 and bus 354 .
  • a motor driver 343 individually controls a plurality of motors 342 according to instructions from the processor 351 .
  • the number of motors 342 is the same as the number of propellers 341 .
  • the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D are fixed to the rotating shaft of each motor 342. Below, if there is no need to distinguish between the first propeller 341A, the second propeller 341B, the third propeller 341C, and the fourth propeller 341D, the first propeller 341A, the second propeller 341B, the third propeller 341C, and the third propeller 341C will be described.
  • the four propellers 341D are called propellers 341, respectively.
  • Each motor 342 rotates the propeller 341 .
  • the aircraft 310 flies by rotating the propellers 341 .
  • the flying object 310 rises, and when the number of rotations per unit time of the plurality of propellers 341 (hereinafter also simply referred to as "rotation number") decreases, the flying object 310 descends.
  • the propulsive forces of the propellers 341 and the gravity acting on the flying object 310 are balanced, the flying object 310 stops in the air (that is, hovers).
  • the aircraft 310 rolls, turns, advances, retreats, and/or laterally moves.
  • the number of the plurality of propellers 341 included in the aircraft main body 320 is four as an example, this is merely an example, and the number of the plurality of propellers 341 may be, for example, three, five or more. It's okay.
  • the imaging device 330 includes an image sensor driver 333, an image sensor 334, an imaging lens 335, a first actuator 336A, a second actuator 336B, a third actuator 336C, a first sensor 337A, a second sensor 337B, a third sensor 337C, and a controller 338.
  • Image sensor driver 333 , image sensor 334 , and controller 338 are connected to processor 351 via input/output I/F 322 and bus 354 .
  • the image sensor driver 333 controls the image sensor 334 according to instructions from the processor 351 .
  • Image sensor 334 is, for example, a CMOS image sensor. Although a CMOS image sensor is exemplified as the image sensor 334 here, the technology of the present disclosure is not limited to this, and other image sensors may be used.
  • the image sensor captures an image of the second subject (for example, the inspection target 3 shown in FIGS. 1 and 2) and outputs the captured image to the processor 351. .
  • the imaging lens 335 has an objective lens 335A, a focus lens 335B, a zoom lens 335C, and an aperture 335D.
  • the objective lens 335A, the focus lens 335B, the zoom lens 335C, and the diaphragm 335D are arranged along the optical axis OA2 of the imaging device 330 from the subject side (object side) to the image sensor 334 side (image side).
  • a lens 335B, a zoom lens 335C, and a diaphragm 335D are arranged in this order.
  • the controller 338 controls the first actuator 336A, the second actuator 336B, and the third actuator 336C according to instructions from the processor 351.
  • Controller 338 is, for example, a device having a computer including a CPU, NVM, RAM, and the like. Although a computer is exemplified here, this is merely an example, and devices including ASIC, FPGA, and/or PLD may be applied. Also, as the controller 338, for example, a device realized by a combination of hardware configuration and software configuration may be used.
  • the first actuator 336A includes a focus slide mechanism (not shown) and a focus motor (not shown).
  • a focus lens 335B is attached to the focus slide mechanism so as to be slidable along the optical axis OA2.
  • a focus motor is connected to the focus slide mechanism, and the focus slide mechanism receives power from the focus motor and operates to move the focus lens 335B along the optical axis OA2.
  • the second actuator 336B includes a zoom slide mechanism (not shown) and a zoom motor (not shown).
  • a zoom lens 335C is attached to the zoom slide mechanism so as to be slidable along the optical axis OA2.
  • a zoom motor is connected to the zoom slide mechanism, and the zoom slide mechanism receives power from the zoom motor to move the zoom lens 335C along the optical axis OA2.
  • an example of a form in which the focus slide mechanism and the zoom slide mechanism are provided separately is given, but this is only an example, and an integrated slide mechanism capable of both focusing and zooming is provided. It may be a mechanism. Also, in this case, power generated by one motor may be transmitted to the slide mechanism without using a focusing motor and a zooming motor.
  • the third actuator 336C includes a power transmission mechanism (not shown) and a diaphragm motor (not shown).
  • the diaphragm 335D has an aperture 335D1, and is a diaphragm with a variable size of the aperture 335D1.
  • the opening 335D1 is formed by a plurality of blades 335D2.
  • the multiple blades 335D2 are connected to the power transmission mechanism.
  • a diaphragm motor is connected to the power transmission mechanism, and the power transmission mechanism transmits the power of the diaphragm motor to the plurality of blades 335D2.
  • the plurality of blades 335D2 change the size of the opening 335D1 by receiving power transmitted from the power transmission mechanism.
  • the diaphragm 335D adjusts exposure by changing the size of the aperture 335D1.
  • the focus motor, zoom motor, and aperture motor are connected to the controller 338, and the controller 338 controls driving of the focus motor, zoom motor, and aperture motor.
  • the controller 338 controls driving of the focus motor, zoom motor, and aperture motor.
  • stepping motors are used for the focus motor, the zoom motor, and the aperture motor. Therefore, the focus motor, zoom motor, and aperture motor operate in synchronization with the pulse signal according to commands from the controller 338 .
  • the first sensor 337A detects the position of the focus lens 335B on the optical axis OA2.
  • An example of the first sensor 337A is a potentiometer.
  • a detection result by the first sensor 337 A is acquired by the controller 338 and output to the processor 351 .
  • the processor 351 adjusts the position of the focus lens 335B on the optical axis OA2 based on the detection result of the first sensor 337A.
  • the second sensor 337B detects the position of the zoom lens 335C on the optical axis OA2.
  • An example of the second sensor 337B is a potentiometer.
  • a detection result by the second sensor 337 B is acquired by the controller 338 and output to the processor 351 .
  • Processor 351 adjusts the position of zoom lens 335C on optical axis OA2 based on the detection result of second sensor 337B.
  • the third sensor 337C detects the size of the opening 335D1.
  • An example of the third sensor 337C is a potentiometer.
  • a detection result by the third sensor 337 ⁇ /b>C is acquired by the controller 338 and output to the processor 351 .
  • Processor 351 adjusts the size of opening 335D1 based on the detection result of third sensor 337C.
  • the storage 52 of the base station 10 stores a flight imaging support program 100 .
  • the processor 51 reads the flight imaging support program 100 from the storage 52 and executes the read flight imaging support program 100 on the RAM 53 .
  • the processor 51 operates as an operation mode setting unit 102 , a flight route setting processing unit 104 , a flight control processing unit 106 and an imaging control processing unit 108 by executing the flight imaging support program 100 .
  • the base station 10 has a flight route setting processing mode, a flight control processing mode, and an imaging control processing mode as operation modes.
  • the operation mode setting unit 102 selectively sets a flight route setting processing mode, a flight control processing mode, and an imaging control processing mode as the operation modes of the base station 10 .
  • the processor 51 operates as the flight route setting processing unit 104 .
  • the processor 51 operates as the flight control processing unit 106 .
  • the processor 51 operates as the imaging control processing unit 108 .
  • the flight route setting processing unit 104 performs flight route setting processing.
  • the flight route setting processing is processing performed by the flight route setting processing unit 104 when the operation mode of the base station 10 is set to the flight route setting processing mode.
  • the flight route setting processing unit 104 includes a first acceptance determination unit 112, a first rotation control unit 114, a first imaging control unit 116, an image information storage control unit 118, a first distance measurement control unit 120, and a distance information storage control unit 122.
  • first zoom magnification determination unit 138 a first zoom magnification storage control section 140 , a first flight route setting section 142 , a second zoom magnification determination section 144 , a second zoom magnification storage control section 146 , and a second flight route setting section 148 .
  • the flight control processing unit 106 performs flight control processing. Flight control processing is processing performed by the flight control processing unit 106 when the operation mode of the base station 10 is set to the flight control processing mode.
  • the flight control processing unit 106 includes a third acceptance determination unit 152, a second imaging control unit 154, an aircraft position derivation unit 156, a position deviation determination unit 158, a second rotation control unit 160, a second ranging control unit 162, a flight It has a body coordinate derivation unit 164 , an imaging position arrival determination unit 166 , a flight instruction generation unit 168 , and a flight instruction transmission control unit 170 .
  • the imaging control processing unit 108 performs imaging control processing.
  • the imaging control processing is processing performed by the imaging control processing unit 108 when the operation mode of the base station 10 is set to the imaging control processing mode.
  • the imaging control processing unit 108 includes a hovering instruction transmission control unit 172, a hovering report reception determination unit 174, a third imaging control unit 176, an aircraft attitude identification unit 178, an attitude correction instruction generation unit 180, an attitude correction instruction transmission control unit 182, Posture correction report reception determination unit 184 , zoom magnification determination unit 186 , first angle of view setting instruction transmission control unit 188 , distance derivation unit 190 , second angle of view setting instruction generation unit 192 , second angle of view setting instruction transmission control unit 194 , a view angle setting report reception determination unit 196 , an imaging instruction transmission control unit 198 , an imaging report reception determination unit 200 , an end determination unit 202 , and an end instruction transmission control unit 204 .
  • the flight imaging program 400 is stored in the storage 352 of the flying object 310 .
  • the processor 351 reads the flight imaging program 400 from the storage 352 and executes the read flight imaging program 400 on the RAM 353 .
  • the processor 351 performs flight imaging processing according to the flight imaging program 400 executed on the RAM 353 .
  • the processor 351 executes a flight instruction reception determination unit 402, a flight control unit 404, a hovering instruction reception determination unit 406, a hovering control unit 408, a hovering report transmission control unit 410, and an attitude correction instruction reception determination.
  • unit 412 posture correction control unit 414, posture correction report transmission control unit 416, view angle setting instruction reception determination unit 418, view angle control unit 420, view angle setting report transmission control unit 422, imaging instruction reception determination unit 424, imaging control It operates as a unit 426 , an image storage control unit 428 , an imaging report transmission control unit 430 , an end instruction reception determination unit 432 , and an end control unit 434 .
  • the inspection object 3 has a wall surface 4 .
  • An example of inspecting the wall surface 4 will be described below as an example.
  • the wall surface 4 is an example of the "surface" according to the technology of the present disclosure.
  • the wall surface 4 has a first surface 4A, a second surface 4B, a third surface 4C, a fourth surface 4D and a fifth surface 4E.
  • the base station 10 is installed at a position where the imaging device 30 can image the wall surface 4 and the rangefinder 40 can measure the distance between the wall surface 4 and the rangefinder 40 .
  • the wall surface 4 is within the distance measurement area of the distance measurement device 40 .
  • the distance measurement area is an area in which the wall surface 4 is scanned multiple times by the distance measurement device 40 while the pedestal 27 is rotated from the first rotation position to the second rotation position. In the distance measurement area, the image of the wall surface 4 is captured by the imaging device 30 in a plurality of times.
  • the first surface 4A, the second surface 4B, the third surface 4C, the fourth surface 4D, and the fifth surface 4E all face the base station 10.
  • the second surface 4B is positioned between the first surface 4A and the third surface 4C.
  • the second surface 4B is inclined with respect to the first surface 4A and the third surface 4C.
  • the second surface 4B is an inclined surface that becomes farther from the base station 10 as it goes from the first surface 4A side toward the third surface 4C side.
  • the third surface 4C is positioned farther from the base station 10 than the first surface 4A.
  • the wall surface 4 of the inspection object 3 has a concave portion 4F.
  • the recess 4F has an opening 4F1 that opens toward the base station 10 side.
  • the area of the opening 4F1 is less than the area that allows the flying object 310 to enter the recess 4F.
  • the recess 4 ⁇ /b>F is formed from the lower end to the upper end of the inspection object 3 .
  • the recess 4F is formed between the third surface 4C and the fifth surface 4E, and the fourth surface 4D is formed by the bottom surface of the recess 4F.
  • the fourth surface 4D is positioned farther from the base station 10 than the third surface 4C and the fifth surface 4E, and the fifth surface 4E is positioned closer to the base station 10 than the third surface 4C. doing.
  • the first surface 4A, the third surface 4C, the fourth surface 4D, and the fifth surface 4E are surfaces parallel to each other. The following description assumes that the first surface 4A, the second surface 4B, the third surface 4C, the fourth surface 4D, and the fifth surface 4E are planes parallel to the vertical direction.
  • the worker 5 gives a measurement start instruction to the reception device 14 .
  • the first reception determination unit 112 determines whether or not the measurement start instruction has been received by the reception device 14 .
  • the first rotation control unit 114 moves the pedestal 27 from the first rotation position to the first rotation with respect to the rotary drive device 20. Control is performed to rotate toward a second rotation position, which is a position different from the position. Specifically, the first rotation control unit 114 rotates the base 27 from the first rotation position toward the second rotation position by operating the pan motor 24 via the motor driver 23 of the rotation drive device 20 . As a result, the imaging device 30 and the distance measuring device 40 attached to the pedestal 27 start rotating in the horizontal direction.
  • the first imaging control unit 116 controls the imaging device 30 to image the wall surface 4 . Specifically, the first imaging control unit 116 causes the image sensor 34 to image the wall surface 4 via the image sensor driver 33 of the imaging device 30 . In this case, the imaging device 30 images a portion of the wall surface 4 in the horizontal direction. As a result, an image is obtained by capturing an image of a portion of the wall surface 4 in the horizontal direction by the imaging device 30 .
  • a rotation detector (not shown) is provided on the pan-tilt mechanism 26 and/or the pedestal 27, and the rotational position of the pedestal 27 (hereinafter also simply referred to as "rotational position") is detected by the rotation detector.
  • the image information storage control unit 118 generates image information based on the image obtained by imaging by the imaging device 30 and the rotational position detected by the rotation detector, and stores the image information in the storage 52 .
  • the image information is information that associates an image obtained by being imaged by the imaging device 30 with the rotational position detected by the rotation detector.
  • the first ranging control unit 120 controls the ranging device 40 to scan the wall surface 4 with laser light. Specifically, the first ranging control unit 120 causes the ranging sensor 44 to output a laser beam by controlling the ranging sensor 44 via the ranging sensor driver 43 of the ranging device 40, and The reflected light of the laser beam reflected by the wall surface 4 is detected by the distance measuring sensor 44 . Further, the first distance measurement control unit 120 rotates the scanner mirror 47 by controlling the scanner actuator 48 via the scanner driver 45 of the distance measurement device 40, thereby changing the position of the laser light in the horizontal direction. In this case, the distance measuring device 40 scans a portion of the wall surface 4 in the horizontal direction.
  • the distance between the wall surface 4 and the distance measuring device 40 is measured by scanning a part of the wall surface 4 in the horizontal direction by the distance measuring device 40 .
  • the distance between the wall surface 4 and the distance measuring device 40 is measured at a plurality of distance measuring points on a portion of the wall surface 4 in the horizontal direction.
  • the distance between the wall surface 4 and the distance measuring device 40 is an example of the "first distance" according to the technology of the present disclosure.
  • the scanner mirror 47 is provided with an angle detector (not shown), and the angle detector detects the rotation angle of the scanner mirror 47 (hereinafter also simply referred to as "rotation angle").
  • the distance information storage control unit 122 generates distance information based on the distance measured for each distance measurement point, the rotational position detected by the rotation detector, and the rotational angle detected by the angle detector, and calculates the distance.
  • the information is stored in storage 52 .
  • the distance information is information that associates the rotational position detected by the rotation detector with respect to the distance measured for each distance measurement location and the rotational angle detected by the angle detector.
  • the rotational position determination unit 124 determines whether or not the rotational position of the pedestal 27 has reached the second rotational position.
  • the rotational position determination unit 124 determines whether or not the rotational position of the base 27 has reached the second rotational position, for example, by comparing the rotational position detected by the rotation detector and the position of the second rotational position. judge.
  • the rotational position determination unit 124 determines that the rotational position of the pedestal 27 has not reached the second rotational position
  • the above-described first imaging control unit 116, image information storage control unit 118, and first ranging control unit 120 is executed.
  • the control by the above-described first imaging control unit 116 and the image information storage control unit 118 is repeatedly executed, so that the first end of the wall surface 4 A plurality of imaging regions of the wall surface 4 are continuously imaged in order from the side to the second end side. Image information corresponding to each imaged area is stored in the storage 52 .
  • the control by the first distance measurement control section 120 and the distance information storage control section 122 is repeatedly executed, thereby the first A plurality of ranging areas of the wall surface 4 are continuously scanned by laser light from the first end side to the second end side. Distance information corresponding to each ranging area is stored in the storage 52 .
  • the rotation stop control unit 126 controls the rotation driving device 20 to stop the rotation of the base 27 when the rotation position determination unit 124 determines that the rotation position of the base 27 has reached the second rotation position. Specifically, the rotation stop control unit 126 stops the rotation of the base 27 by stopping the rotation of the pan motor 24 via the motor driver 23 of the rotation drive device 20 .
  • the imaging device 30 captures images of the wall surface 4 a plurality of times
  • the distance measuring device 40 captures the wall surface 4 a plurality of times.
  • Image information and distance information corresponding to the wall surface 4 are obtained by scanning the wall surface.
  • the image display control unit 128 displays an image (that is, an image in which the wall surface 4 appears as an image) on the display 16 based on the image information stored in the storage 52. is displayed.
  • the image display control unit 128, based on the rotational positions included in the image information corresponding to the first surface 4A, the second surface 4B, the third surface 4C, the fourth surface 4D, and the fifth surface 4E , the first surface 4A, the second surface 4B, the third surface 4C, the fourth surface 4D, and the fifth surface 4E are displayed side by side on the display 16. .
  • the operator 5 Based on the image displayed on the display 16 (for example, while visually referring to the image), the operator 5 performs the first surface 4A, the second surface 4B, the third surface 4C, the fourth surface 4D, and the fifth surface.
  • An inspection target surface 4G to be inspected by the flying object 310 is determined from the surface 4E.
  • the worker 5 provides the reception device 14 with inspection target surface designation information indicating that the inspection target surface 4G is designated.
  • the second reception determination unit 130 determines whether or not the inspection target surface specification information has been received by the reception device 14 .
  • the copying surface setting unit 132 sets the copying surface 6 based on the inspection surface designation information.
  • the copying surface 6 is a surface that is separated from the inspection target surface 4G in the normal direction of the inspection target surface 4G by a predetermined distance L and that follows the inspection target surface 4G (that is, a virtual surface along the inspection target surface 4G).
  • the predetermined distance L is a distance at which the inspection target surface 4G is included in the depth of field of the imaging device 330 of the aircraft 310, and is a preset distance. As an example, the default distance L is set to 1 m to 3 m.
  • the operator 5 designates the first surface 4A, the second surface 4B, and the third surface 4C as the inspection target surface 4G. Therefore, in the example shown in FIG. 16, the copying surface 6 has a first copying surface 6A that copies the first surface 4A, a second copying surface 6B that copies the second surface 4B, and a third copying surface 6C that copies the third surface 4C. is set by the copying plane setting unit 132 .
  • the first copying surface 6A is a surface separated from the first surface 4A by a predetermined distance L
  • the second copying surface 6B is a surface separated from the second surface 4B by a predetermined distance L
  • the third copying surface 6C is , a plane separated by a predetermined distance L from the third plane 4C.
  • the smooth surface setting unit 134 smoothes the copying surface 6 to set the smooth surface 7 (that is, the smooth virtual surface facing the wall surface 4).
  • “Smooth” refers to, for example, a smooth aspect without discontinuities and without irregularities. Also, “smoothing” is realized by reducing the degree of bending of the copying surface 6 to a degree designated as an allowable degree.
  • smoothing the copying surface 6 means replacing the copying surface 6 with the smooth surface 7 as it is.
  • the smooth surface setting unit 134 sets the smooth surface 7 that satisfies the following first and second conditions.
  • the first condition is a condition that the smooth surface 7 is a surface that passes through at least one of the plurality of surfaces forming the copying surface 6 and faces the inspection target surface 4G.
  • the second condition is a condition that the smooth surface 7 is defined as a surface having a predetermined distance L or more between each of the plurality of surfaces forming the inspection target surface 4G and the smooth surface 7 .
  • the smooth surface 7 that satisfies the above first and second conditions passes through the first copying surface 6A, the second copying surface 6B, and the third copying surface 6C, and the inspection target surface 4G.
  • Opposing smooth surfaces 7 are provided.
  • the example shown in FIG. 17 is an example in which the operator 5 designates the third surface 4C, the fourth surface 4D, and the fifth surface 4E as the inspection target surface 4G.
  • a copying surface 6 having a third copying surface 6C that copies the third surface 4C, a fourth copying surface 6D that copies the fourth surface 4D, and a fifth copying surface 6E that copies the fifth surface 4E is copied.
  • the third copying surface 6C is a surface separated from the third surface 4C by a predetermined distance L
  • the fourth copying surface 6D is a surface separated from the fourth surface 4D by a predetermined distance L
  • the fifth copying surface 6E is , and a plane separated by a predetermined distance L from the fifth plane 4E.
  • the smooth surface 7 that passes through the fifth copying surface 6E and faces the inspection target surface 4G is set as the smooth surface 7 that satisfies the above first and second conditions.
  • the distance determination unit 136 determines whether the distance between the inspection target surface 4G and the smooth surface 7 is constant. For example, in the example shown in FIG. 16, the distance between the surface to be inspected 4G and the smooth surface 7 is the predetermined distance L, which is constant. Therefore, in the example shown in FIG. 16, the distance determination unit 136 determines that the distance between the inspection target surface 4G and the smooth surface 7 is constant. On the other hand, for example, in the example shown in FIG. 17, the distance between the inspection target surface 4G and the smooth surface 7 is not constant.
  • the distance determination unit 136 determines that the distance between the inspection target surface 4G and the smooth surface 7 is not constant.
  • the example shown in FIG. 18 is an example in which the distance between the inspection target surface 4G and the smooth surface 7 is constant at the predetermined distance L, similar to the example shown in FIG.
  • the first zoom magnification determination unit 138 determines that the distance between the inspection target surface 4G and the smooth surface 7 is constant when the distance determination unit 136 determines that the distance is constant.
  • the zoom magnification of the imaging device 330 (see FIG. 1) of the flying object 310 is determined to be the first zoom magnification.
  • the first zoom magnification is a zoom magnification at which the pixel resolution of the imaging device 330 is the default value when the imaging device 330 captures an image of the inspection target surface 4G from a position a predetermined distance L away from the inspection target surface 4G. be.
  • the pixel resolution of the imaging device 330 corresponds to the size of the field of view per pixel of the image sensor 334 provided in the imaging device 330 .
  • the size of the field of view corresponds to the range in which the subject is actually imaged.
  • the default value for the pixel resolution is the presence or absence of damage to the inspection target surface 4G when image analysis processing is performed by the image analysis device 2 (see FIG. 1) on an image obtained by imaging the inspection target surface 4G. and/or is set to a value that allows inspection of the degree of damage or the like.
  • the first zoom magnification storage control unit 140 causes the storage 52 to store the first zoom magnification determined by the first zoom magnification determination unit 138 .
  • the first flight route setting unit 142 sets a plurality of imaging positions 8A on the smooth surface 7 based on the first zoom magnification determined by the first zoom magnification determining unit 138, thereby determining the plurality of imaging positions 8A.
  • a flight route 8 to pass through is set.
  • a plurality of imaging positions 8A are positions where the inspection target surface 4G is imaged by the imaging device 330 (see FIG. 1) of the flying object 310.
  • the first flight route setting unit 142 selects adjacent imaging positions 8A among the plurality of imaging positions 8A.
  • a flight route 8 passing through the plurality of imaging positions 8A is set.
  • a plurality of imaging positions 8A can be obtained as described later. The images obtained by being imaged by the imaging device 330 each time the position 8A is reached partially overlap each other.
  • a plurality of imaging positions 8A is an example of a "first imaging position" according to the technology of the present disclosure.
  • the example shown in FIG. 19 is an example in which the distance between the inspection target surface 4G and the smooth surface 7 is not constant, similar to the example shown in FIG.
  • the second zoom magnification determining unit 144 determines that the distance between the inspection target surface 4G and the smooth surface 7 is not constant by the distance determining unit 136.
  • the zoom magnification of the imaging device 330 (see FIG. 1) of the flying object 310 is determined to be the second zoom magnification.
  • the second zoom magnification is set by the imaging device 330 from a position separated by the shortest distance between the inspection target surface 4G and the smooth surface 7 (in this case, the distance L5 between the fifth surface 4E and the smooth surface 7). is the zoom magnification at which the pixel resolution of the imaging device 330 becomes the above-described default value when the inspection target surface 4G is imaged by .
  • the second zoom magnification storage control unit 146 causes the storage 52 to store the second zoom magnification determined by the second zoom magnification determination unit 144 .
  • the second flight route setting unit 148 sets a plurality of imaging positions 8A on the smooth surface 7 based on the second zoom magnification determined by the second zoom magnification determining unit 144, thereby determining the plurality of imaging positions 8A.
  • a flight route 8 to pass through is set.
  • the second zoom magnification determined by the second zoom magnification determination unit 144 is used to image the inspection target surface 4G. Control is performed to keep the pixel resolution of the imaging device 330 constant by adjusting according to the distance from the position 8A.
  • the second flight route setting unit 148 adjusts the second zoom magnification determined by the second zoom magnification determination unit 144 according to the distance between the inspection target surface 4G and the imaging position 8A.
  • a flight route passing through the plurality of imaging positions 8A can be obtained.
  • Set 8 By setting a plurality of imaging positions 8A at positions where the imaging ranges 331 of the imaging device 330 partly overlap with each other at adjacent imaging positions 8A among the plurality of imaging positions 8A, a plurality of imaging positions 8A can be obtained as described later. The images obtained by being imaged by the imaging device 330 each time the position 8A is reached partially overlap each other.
  • the flying object 310 is placed within the imaging range 31 of the imaging device 30 of the base station 10 .
  • the operator 5 issues a flight start instruction to the reception device 14 when the aircraft 310 is ready to start flying.
  • the third reception determination unit 152 determines whether or not the flight start instruction has been received by the reception device 14 .
  • the second imaging control unit 154 controls the imaging device 30 to capture an imaging scene including the aircraft 310 . conduct. Specifically, the second imaging control unit 154 causes the image sensor 34 to image the imaging scene including the aircraft 310 via the image sensor driver 33 of the imaging device 30 . As a result, an image is obtained by capturing an imaging scene including the flying object 310 with the imaging device 30 .
  • the image obtained by imaging the imaging scene including the flying object 310 is an example of the "second image" according to the technology of the present disclosure.
  • the flying object position derivation unit 156 performs object recognition processing on an image obtained by capturing an imaging scene including the flying object 310 with the imaging device 30, thereby determining the flying object 310 included as an image in the image. Derive the position in the image of
  • the positional deviation determination unit 158 determines whether the position of the flying object 310 is shifted with respect to the central portion of the angle of view of the imaging device 30. determine whether or not
  • the second rotation control unit 160 controls the rotation angle in the horizontal direction and/or the vertical direction of the rotary drive device 20. is adjusted to an angle at which the flying object 310 is positioned at the center of the angle of view of the imaging device 30 .
  • the second rotation control unit 160 controls the pan motor 24 and/or the tilt motor 25 via the motor driver 23 of the rotary drive device 20, thereby controlling the horizontal rotation angle and/or the rotation angle of the rotary drive device 20.
  • the rotation angle in the vertical direction is adjusted so that the flying object 310 is positioned at the center of the angle of view of the imaging device 30 .
  • the flying object 310 is included in the central portion of the ranging range 41 (see FIG. 21) of the ranging device 40 .
  • the horizontal rotation angle and/or the vertical rotation angle of the rotary drive device 20 will be referred to as the rotation angle of the rotary drive device 20 .
  • the rotation angle of the rotary drive device 20 in this case is an example of the "second rotation angle" according to the technology of the present disclosure.
  • the second ranging control unit 162 controls the ranging device 40 to scan the ranging range 41 with laser light.
  • the second ranging control unit 162 controls the ranging sensor 44 via the ranging sensor driver 43 of the ranging device 40 to cause the ranging sensor 44 to output a laser beam, and
  • the distance measurement sensor 44 is made to detect the reflected light of the laser light reflected by the object (for example, the flying object 310 and other objects in this case) included in the distance measurement range 41 .
  • the second distance measurement control unit 162 rotates the scanner mirror 47 by controlling the scanner actuator 48 via the scanner driver 45 of the distance measurement device 40, thereby changing the position of the laser light in the horizontal direction.
  • the distance measuring range 41 is scanned by the distance measuring device 40 .
  • the distance between the object and the ranging device 40 is measured.
  • the distance between the object and the range finder 40 is measured at a plurality of range measurement locations in the range 41 .
  • the distance between the flying object 310 and the ranging device 40 is measured by the ranging device 40 .
  • the flying object coordinate deriving unit 164 calculates the absolute coordinates of the rotation driving device 20, the rotation angle of the rotation driving device 20, the angle of the laser beam irradiated from the rangefinder 40 toward the flying object 310, and the flying object 310 and the measured value. Based on the distance to the range device 40, the absolute coordinates of the aircraft 310 are derived. Absolute coordinates are coordinates measured from the origin of a coordinate system (here, for example, an absolute coordinate system set at a fixed point on the imaging system S).
  • the absolute coordinates of the rotary drive device 20 are an example of the "first absolute coordinates" according to the technology of the present disclosure
  • the absolute coordinates of the aircraft 310 are an example of the "second absolute coordinates" according to the technology of the present disclosure. .
  • the flying object coordinate derivation unit 164 calculates the absolute coordinates of the rotary drive device 20, the rotation angle of the rotary drive device 20, the angle of the laser light emitted from the rangefinder 40 toward the flying object 310, and the flight
  • the distance between body 310 and rangefinder 40 is obtained as follows.
  • the flying object coordinate derivation unit 164 acquires the distance between the flying object 310 and the ranging device 40 from the distance information obtained by scanning the ranging range 41 with the ranging device 40 .
  • the flying object coordinate deriving unit 164 acquires the distance measured about the central portion of the ranging range 41 of the ranging device 40 as the distance between the flying object 310 and the ranging device 40 .
  • the distance between the vehicle 310 and the rangefinder 40 corresponds to the distance between the vehicle 310 and the LiDAR scanner.
  • the flying object coordinate derivation unit 164 calculates the average value of the distances measured at a plurality of distance measurement points in a predetermined area including the center of the ranging range 41 of the ranging device 40, It may be obtained as the distance between
  • the default area is, for example, an area that includes only the flying object 310 .
  • the distance between the flying object 310 and the rangefinder 40 is an example of the "second distance" according to the technology of the present disclosure.
  • the aircraft coordinate derivation unit 164 may, for example, calculate the coordinates of the base station 10 (for example, Absolute coordinates of the rotary drive device 20 are obtained based on the three-dimensional coordinates corresponding to latitude, longitude, and altitude.
  • the absolute coordinates of rotary drive device 20 correspond to the absolute coordinates of base station 10 .
  • the flying object coordinate derivation unit 164 acquires the angle of the laser beam emitted from the rangefinder 40 toward the flying object 310 based on the rotation angle of the scanner mirror 47 detected by the angle detector.
  • the angle of the laser light emitted from the rangefinder 40 toward the flying object 310 corresponds to the angle of the laser light emitted from the LiDAR scanner toward the flying object 310 .
  • the aircraft coordinate derivation unit 164 calculates the rotation angle of the rotary drive device 20 based on the rotation position of the base 27 detected by the pan/tilt mechanism 26 and/or a rotation detector (not shown) provided on the base 27. get.
  • the imaging position arrival determination unit 166 determines the coordinates of the flying object 310 derived by the flying object coordinates deriving unit 164, and the imaging position 8A closest to the flying object 310 among the plurality of imaging positions 8A (hereinafter referred to as the target imaging position 8A). ), it is determined whether or not the flying object 310 has reached the target imaging position 8A.
  • the flight instruction generating unit 168 combines the coordinates of the flying object 310 derived by the flying object coordinates deriving unit 164 with the target.
  • a flight instruction for the flying object 310 is generated based on the difference between the coordinates of the imaging position 8A.
  • the flight instruction generating unit 168 determines whether the flying object 310 flies along the flight route 8 based on the absolute coordinates of the flying object 310 derived by the flying object coordinate deriving unit 164 and the absolute coordinates of the target imaging position 8A.
  • the flight instruction generator 168 calculates the rotation speed of each propeller 341 corresponding to the flight direction of the aircraft 310 and the amount of movement of the aircraft 310, and generates flight instructions corresponding to the rotation speed of each propeller 341. .
  • the flight instruction transmission control unit 170 controls transmission of flight instructions to the aircraft 310 via the communication device 12 .
  • the flight instruction reception determination unit 402 determines whether the communication device 312 has received the flight instruction.
  • the flight control unit 404 controls the flight device 340 according to the flight instruction. Specifically, the flight control unit 404 controls the plurality of motors 342 via the motor driver 343 of the flight device 340 according to the flight instruction, thereby adjusting the rotation speed of each propeller 341 to the rotation speed corresponding to the flight instruction. do. As a result, the flying object 310 flies toward the target imaging position 8A.
  • the hovering instruction transmission control unit 172 causes the communication device 12 to move when the imaging position arrival determination unit 166 determines that the aircraft 310 has reached the target imaging position 8A. Control is performed to transmit a hovering instruction to the flying object 310 via.
  • the hovering instruction reception determining unit 406 determines whether the communication device 312 has received the hovering instruction.
  • the hovering control unit 408 controls the flight device 340 to hover the aircraft 310 . Specifically, the hovering control unit 408 controls the plurality of motors 342 via the motor driver 343 of the flight device 340 to adjust the rotation speed of each propeller 341 to the rotation speed at which the aircraft 310 hovers. As a result, the flying object 310 hovers.
  • the hovering report transmission control unit 410 performs control to transmit a hovering report to the effect that the flying object 310 is hovering to the base station 10 via the communication device 312 after the control by the hovering control unit 408 is performed. conduct.
  • the hovering report reception determining unit 174 determines whether or not the communication device 12 has received the hovering report.
  • the third imaging control unit 176 controls the imaging device 30 to capture an imaging scene including the flying object 310 . Specifically, the third imaging control unit 176 causes the image sensor 34 to image the imaging scene including the flying object 310 via the image sensor driver 33 of the imaging device 30 . As a result, an image is obtained by capturing an imaging scene including the flying object 310 with the imaging device 30 .
  • the flying object posture identification unit 178 performs object recognition processing (for example, template matching type object recognition processing, or , AI-based object recognition processing), the attitude of the flying object 310 is specified based on the positions of the plurality of propellers 341 shown in the image. Specifically, the aircraft attitude identification unit 178 identifies the positions of the plurality of propellers 341 by identifying the colors of the plurality of propellers 341 shown in the image based on the image. Then, the flying object attitude identification unit 178 identifies the attitude of the flying object 310 based on the positions of the propellers 341 .
  • the attitude of the aircraft 310 includes the orientation of the aircraft 310 and/or the inclination of the aircraft 310 and the like.
  • the attitude correction instruction generating unit 180 generates an attitude correction instruction for the flying object 310 based on the attitude of the flying object 310 identified by the flying object attitude identifying unit 178 . Specifically, based on the attitude of the flying object 310 identified by the flying object attitude identifying unit 178, the attitude correction instruction generating unit 180 adjusts the attitude of the flying object 310 to a horizontal state facing the inspection target surface 4G. A posture correction amount for correcting the posture is calculated. Then, the attitude correction instruction generator 180 calculates the number of rotations of each propeller 341 corresponding to the amount of attitude correction, and generates an attitude correction instruction corresponding to the number of rotations of each propeller 341 .
  • the attitude correction instruction transmission control unit 182 controls transmission of the attitude correction instruction to the aircraft 310 via the communication device 12 .
  • the attitude correction instruction reception determination unit 412 determines whether the communication device 312 has received the attitude correction instruction.
  • the attitude correction control unit 414 instructs the flight device 340 to correct the attitude of the aircraft 310 according to the attitude correction instruction. do. Specifically, the attitude correction control unit 414 controls the plurality of motors 342 via the motor driver 343 of the flight device 340 in accordance with the attitude correction instruction, thereby increasing the number of rotations of the plurality of propellers 341 to the number of revolutions corresponding to the attitude correction instruction. Adjust the rpm. As a result, the attitude of the flying object 310 is corrected so that it faces the inspection target surface 4G in a horizontal state. By correcting the posture of the flying object 310 to face the inspection target surface 4G in a horizontal state, the optical axis OA2 of the imaging device 330 is perpendicular to the inspection target surface 4G when the imaging device 330 is horizontal.
  • the attitude correction report transmission control unit 416 sends an attitude correction report to the base station 10 via the communication device 312 to the effect that the attitude of the aircraft 310 has been corrected. Control to send.
  • the distance between the inspection target surface 4G and the smooth surface 7 is constant at the predetermined distance L, as in the example shown in FIG.
  • the first zoom magnification is stored in the storage 52 by (see FIG. 18).
  • the posture correction report reception determining unit 184 determines whether or not the communication device 12 has received the posture correction report.
  • the zoom magnification determination unit 186 performs the above-described first zoom magnification storage control unit 140 or the second zoom magnification storage control unit 146. determines whether the zoom magnification stored in the storage 52 is the first zoom magnification or the second zoom magnification.
  • the zoom magnification determination unit 186 determines that the zoom magnification stored in the storage 52 is the first zoom magnification
  • the first angle-of-view setting instruction transmission control unit 188 sends a command to the aircraft 310 via the communication device 12. control to transmit the first angle of view setting instruction corresponding to the first zoom magnification.
  • the distance between the inspection target surface 4G and the smooth surface 7 is not constant as in the example shown in FIG. ) is an example in which the second zoom magnification is stored in the storage 52 .
  • the distance deriving unit 190 determines that the zoom magnification stored in the storage 52 is the second zoom magnification, if the zoom magnification determination unit 186 determines that the zoom magnification is the second zoom magnification. Based on the distance information stored in the storage 52 by the information storage controller 122, the distance between the inspection target surface 4G and the target imaging position 8A is derived.
  • the second angle-of-view setting instruction generation unit 192 Based on the distance derived by the distance derivation unit 190, the second angle-of-view setting instruction generation unit 192 adjusts the second zoom magnification to the zoom magnification that makes the pixel resolution of the imaging device 330 the default value described above. Then, the second angle-of-view setting instruction generating section 192 generates a second angle-of-view setting instruction corresponding to the second zoom magnification adjusted based on the distance derived by the distance deriving section 190 .
  • the second angle-of-view setting instruction generation unit 192 sets the second zoom magnification A second angle of view setting instruction corresponding to the second zoom magnification determined by the determining unit 144 is generated.
  • the second view angle setting instruction generation unit 192 determines the second zoom magnification determination unit The second zoom magnification is adjusted by increasing the second zoom magnification determined by 144 according to the distance derived by the distance derivation unit 190 . Then, the second angle-of-view setting instruction generation unit 192 generates a second angle-of-view setting instruction corresponding to the adjusted second zoom magnification.
  • the second angle of view setting instruction transmission control unit 194 performs control to transmit the second angle of view setting instruction generated by the second angle of view setting instruction generation unit 192 to the aircraft 310 via the communication device 12 .
  • the first angle of view setting instruction and the second angle of view setting instruction will be referred to as the angle of view setting instruction when there is no need to distinguish between the first angle of view setting instruction and the second angle of view setting instruction.
  • the view angle setting instruction reception determining unit 418 determines whether the communication device 312 has received the view angle setting instruction.
  • the angle-of-view control unit 420 instructs the imaging device 330 to set the angle of view of the imaging device 330 . Control is performed to set the angle of view corresponding to the instruction. Specifically, the view angle control unit 420 controls the second actuator 336B via the controller 338 to adjust the position of the zoom lens 335C to a position corresponding to the view angle setting instruction. The zoom magnification of the imaging device 330 is adjusted by adjusting the position of the zoom lens 335C.
  • the angle-of-view control unit 420 sets the zoom magnification of the imaging device 330 to the first zoom magnification in accordance with the first angle-of-view setting instruction.
  • the angle-of-view control unit 420 sets the zoom magnification of the imaging device 330 to the second zoom magnification in accordance with the second angle-of-view setting instruction.
  • the view angle control unit 420 controls the first actuator 336A via the controller 338 to adjust the position of the focus lens 335B to a position corresponding to the view angle setting instruction.
  • the focus of the imaging device 330 is adjusted by adjusting the focus position.
  • the view angle control section 420 may operate at least one of the zoom lens 335C and the focus lens 335B.
  • the angle-of-view setting report transmission control unit 422 After the control by the angle-of-view control unit 420, the angle-of-view setting report transmission control unit 422 notifies the base station 10 via the communication device 312 that the angle of view of the imaging device 330 corresponds to the angle-of-view setting instruction. Control is performed to transmit a field angle setting report to the effect that the field angle has been set.
  • the view angle setting report reception determination unit 196 determines whether the communication device 12 has received the view angle setting report.
  • the image capture command transmission control unit 198 transmits the capture command to the aircraft 310 via the communication device 12. to control.
  • the imaging instruction reception determination unit 424 determines whether or not the communication device 312 has received the imaging instruction.
  • the imaging control unit 426 controls the imaging device 330 to image the inspection target surface 4G. Specifically, the imaging control unit 426 causes the image sensor 334 to image the inspection target surface 4G via the image sensor driver 333 of the imaging device 330 . In this case, the imaging device 330 images a part of the inspection target surface 4G. As a result, an image is obtained by capturing an image of a portion of the inspection target surface 4G by the imaging device 330 . An image obtained by being captured by the imaging device 330 under the control of the imaging control unit 426 is an example of the "first image" according to the technology of the present disclosure.
  • the image storage control unit 428 causes the image memory 314 to store an image captured by the imaging device 330 .
  • the imaging report transmission control unit 430 sends an imaging report to the base station 10 via the communication device 312 to the effect that part of the inspection target surface 4G has been imaged by the imaging device 330. to control the transmission of
  • the imaging report reception determination unit 200 determines whether or not the communication device 12 has received the imaging report.
  • the termination determination unit 202 determines whether or not the conditions for terminating the flight imaging support process are satisfied.
  • a condition for ending the flight imaging support process is that the number of imaging reports reaches the number of imaging positions 8A. If the number of imaging reports is smaller than the number of imaging positions 8A, the termination determination unit 202 determines that the conditions for terminating the flight imaging support process are not met.
  • the flying object 310 moves to each imaging position 8A in order by flying along the flight route 8, and each time it reaches each of the plurality of imaging positions 8A.
  • a plurality of images are acquired by imaging the surface 4G to be inspected by the imaging device 330 at the time.
  • the zoom magnification of the imaging device 330 is maintained at the first zoom magnification at each imaging position 8A. Accordingly, the pixel resolution of the imaging device 330 is kept constant.
  • the distance between the inspection target surface 4G and each imaging position 8A fluctuates.
  • the pixel resolution of the imaging device 330 is kept constant by adjusting the second zoom magnification of the imaging device 330 according to the distance between the inspection target surface 4G and the imaging position 8A.
  • the range actually captured by the imaging device 330 is kept constant.
  • the distance between the inspection target surface 4G and the imaging position 8A corresponds to the distance between the inspection target surface 4G and the imaging device 330 .
  • the termination determination unit 202 determines that the conditions for terminating the flight imaging support process have been met.
  • the termination instruction transmission control unit 204 performs control to transmit an termination instruction to the flying object 310 via the communication device 12 when the termination determination unit 202 determines that the condition for terminating the flight imaging support processing is satisfied. .
  • the termination instruction reception determination unit 432 determines whether the communication device 312 has received the termination instruction.
  • the termination control section 434 controls the flight device 340 to terminate the flight.
  • the control to end the flight includes, for example, control to land the flying object 310, control to return the flying object 310 to the position where the flying object 310 started the flight imaging process, and/or control the flying object 310 to a pilot (not shown). ), and the like.
  • the termination control unit 434 adjusts the rotation speed of each propeller 341 by controlling a plurality of motors 342 via the motor driver 343 of the flight device 340 according to the termination instruction.
  • FIG. 34 to 42 the action of the imaging system S according to the first embodiment will be described with reference to FIGS. 34 to 42.
  • FIG. 34 An example of the flow of flight imaging support processing performed by the processor 51 of the base station 10 will be described with reference to FIGS. 34 to 39.
  • FIG. 34 An example of the flow of flight imaging support processing performed by the processor 51 of the base station 10 will be described with reference to FIGS. 34 to 39.
  • step ST10 the operation mode setting unit 102 sets the operation mode of the base station 10 to the flight route setting mode. After the process of step ST10 is executed, the flight imaging support process proceeds to step ST11.
  • step ST11 the first reception determination unit 112 determines whether or not the reception device 14 has received the measurement start instruction. In step ST11, if the measurement start instruction has not been received by the receiving device 14, the determination is negative, and the determination of step ST11 is performed again. In step ST11, when the measurement start instruction is accepted by the accepting device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST12.
  • step ST12 the first rotation control section 114 rotates the pedestal 27 from the first rotation position toward the second rotation position by controlling the rotary drive device 20 based on the measurement start instruction.
  • step ST13 the flight imaging support process proceeds to step ST13.
  • step ST13 the first imaging control unit 116 causes the imaging device 30 to image the wall surface 4.
  • step ST13 the flight imaging support process proceeds to step ST14.
  • step ST14 the image information storage control unit 118 causes the storage 52 to store image information generated by associating the rotation position of the base 27 with the image obtained at step ST13.
  • step ST14 the flight imaging support process proceeds to step ST15.
  • step ST15 the first ranging control unit 120 causes the ranging device 40 to scan the wall surface 4. After the process of step ST15 is executed, the flight imaging support process proceeds to step ST16.
  • step ST16 the distance information storage control unit 122 adds the rotational position detected by the rotation detector (not shown) and the rotational angle detected by the angle detector (not shown) to the distance measured in step ST15.
  • the storage 52 stores the distance information generated by associating the .
  • step ST17 the rotational position determination unit 124 determines whether or not the rotational position of the pedestal 27 has reached the second rotational position. In step ST17, if the rotational position of the pedestal 27 has not reached the second rotational position, the determination is negative, and the flight imaging support process proceeds to step ST13.
  • step ST13 and ST14 are repeatedly executed, so that from the first end side to the second end side of the wall surface 4, A plurality of areas to be imaged on the wall surface 4 are continuously imaged. Image information corresponding to each imaged area is stored in the storage 52 .
  • the steps ST15 and ST16 described above are repeatedly executed, thereby moving the wall surface 4 from the first end side to the second end side. A plurality of range-finding areas on the wall surface 4 are successively scanned by the laser light. Distance information corresponding to each ranging area is stored in the storage 52 .
  • step ST17 when the rotational position of the pedestal 27 reaches the second rotational position, the determination is affirmative, and the flight imaging support process proceeds to step ST18.
  • step ST18 the rotation stop control section 126 stops the rotation of the pedestal 27 by stopping the rotation of the rotary drive device 20.
  • the flight imaging support process proceeds to step ST20 shown in FIG.
  • step ST20 shown in FIG. 35 the image display control unit 128 causes the display 16 to display an image based on the image information stored in the storage 52.
  • the wall surface 4 is represented as an image in the image.
  • step ST21 the second reception determination unit 130 determines whether or not the inspection target surface designation information provided by the operator 5 has been received by the reception device 14. In step ST21, if the inspection target surface specification information has not been received by the receiving device 14, the determination is negative, and the determination of step ST21 is performed again. In step ST21, when the inspection target plane designation information is received by the receiving device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST22.
  • step ST22 the copying plane setting unit 132 sets the copying plane 6 that copies the inspection plane 4G based on the inspection plane designation information. After the process of step ST22 is executed, the flight imaging support process proceeds to step ST23.
  • step ST23 the smooth surface setting unit 134 sets the smooth surface 7 by smoothing the copying surface 6. After the process of step ST23 is executed, the flight imaging support process proceeds to step ST24.
  • step ST24 the distance determination unit 136 determines whether or not the distance between the inspection target surface 4G and the smooth surface 7 is constant based on the distance information stored in the storage 52. In step ST24, if the distance between the inspection target surface 4G and the smooth surface 7 is constant, the determination is affirmative, and the flight imaging support process proceeds to step ST25. In step ST24, if the distance between the inspection target surface 4G and the smooth surface 7 is not constant, the determination is negative, and the flight imaging support process proceeds to step ST28.
  • the first zoom magnification determining unit 138 determines the zoom magnification of the imaging device 330 of the flying object 310 to be the first zoom magnification.
  • the first zoom magnification is a zoom magnification at which the pixel resolution of the imaging device 330 is a default value.
  • step ST26 the first zoom magnification storage control section 140 causes the storage 52 to store the first zoom magnification determined by the first zoom magnification determination section 138.
  • FIG. After the process of step ST26 is executed, the flight imaging support process proceeds to step ST27.
  • step ST27 the first flight route setting section 142 sets a plurality of imaging positions 8A on the smooth surface 7 based on the first zoom magnification determined by the first zoom magnification determining section 138, thereby obtaining a plurality of A flight route 8 passing through the imaging position 8A is set.
  • the first flight route setting unit 142 images the inspection target surface 4G at the first zoom magnification determined by the first zoom magnification determining unit 138
  • the first flight route setting unit 142 selects adjacent imaging positions 8A among the plurality of imaging positions 8A.
  • a flight route 8 passing through the plurality of imaging positions 8A is set.
  • step ST28 the second zoom magnification determination unit 144 determines the zoom magnification of the imaging device 330 of the flying object 310 to be the second zoom magnification. After the process of step ST28 is executed, the flight imaging support process proceeds to step ST29.
  • step ST29 the second zoom magnification storage control section 146 stores the second zoom magnification determined by the second zoom magnification determination section 144 in the storage 52. After the process of step ST29 is executed, the flight imaging support process proceeds to step ST30.
  • step ST30 the second flight route setting section 148 sets a plurality of imaging positions 8A on the smooth surface 7 based on the second zoom magnification determined by the second zoom magnification determination section 144, thereby obtaining a plurality of A flight route 8 passing through the imaging position 8A is set. Even when the second zoom magnification is adjusted according to the distance between the inspection target surface 4G and the imaging position 8A in steps ST73 and ST74, which will be described later, the second flight route setting unit 148 sets a plurality of imaging positions 8A. By setting a plurality of imaging positions 8A at positions where the imaging ranges 331 of the imaging devices 330 partly overlap with each other at adjacent imaging positions 8A, a flight route 8 passing through the plurality of imaging positions 8A is set. After the process of step ST30 is executed, the flight imaging support process proceeds to step ST40 shown in FIG.
  • step ST40 shown in FIG. 36 the operation mode setting section 102 sets the operation mode of the base station 10 to the flight control mode. After the process of step ST40 is executed, the flight imaging support process proceeds to step ST41.
  • step ST41 the third reception determination unit 152 determines whether or not the reception device 14 has received the flight start instruction. In step ST41, if the instruction to start flight has not been received by the receiving device 14, the determination is negative, and the determination in step ST41 is performed again. In step ST41, when the instruction to start flight is accepted by the accepting device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST42.
  • step ST42 the second imaging control unit 154 causes the imaging device 30 to capture an imaging scene including the flying object 310.
  • the flight imaging support process proceeds to step ST43.
  • step ST43 the flying object position derivation unit 156 derives the position of the flying object 310 in the image obtained by being imaged by the imaging device 30. After the process of step ST43 is executed, the flight imaging support process proceeds to step ST44.
  • step ST44 based on the position of the flying object 310 in the image derived in step ST43, the positional deviation determination unit 158 determines whether the position of the flying object 310 is shifted from the central portion of the angle of view of the imaging device 30. determine whether or not In step ST44, if the flying object 310 is out of position with respect to the central portion of the angle of view, the determination is affirmative, and the flight imaging support process proceeds to step ST45. In step ST44, if the position of the flying object 310 is not shifted with respect to the central portion of the angle of view, the determination is negative, and the flight imaging support process proceeds to step ST46.
  • step ST45 the second rotation control unit 160 adjusts the rotation angle of the rotation drive device 20 to an angle at which the flying object 310 is positioned at the center of the angle of view of the imaging device 30.
  • step ST45 the flight imaging support process proceeds to step ST46.
  • step ST46 the second ranging control section 162 causes the ranging device 40 to scan the ranging range 41 with laser light. In this case, since the flying object 310 is positioned within the ranging range 41 of the ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained. After the process of step ST46 is executed, the flight imaging support process proceeds to step ST47.
  • step ST47 the flying object coordinate derivation unit 164 determines the absolute coordinates of the rotary drive device 20, the rotation angle of the rotary drive device 20, the angle of the laser light emitted from the rangefinder 40 toward the flying object 310, and the flight Based on the distance between the body 310 and the rangefinder 40, the absolute coordinates of the aircraft 310 are derived.
  • the flight imaging support process proceeds to step ST48.
  • step ST48 the imaging position arrival determination unit 166 determines whether the flying object 310 has reached the target imaging position 8A based on the absolute coordinates of the flying object 310 and the absolute coordinates of the target imaging position 8A derived in step ST47. determine whether In step ST48, if the flying object 310 has not reached the target imaging position 8A, the determination is negative, and the flight imaging support process proceeds to step ST49. In step ST48, when the flying object 310 reaches the target imaging position 8A, the determination is affirmative, and the flight imaging support process proceeds to step ST60 shown in FIG.
  • step ST49 the flight instruction generating unit 168 generates flight instructions for the flying object 310 based on the difference between the absolute coordinates of the flying object 310 derived at step ST47 and the absolute coordinates of the target imaging position 8A. After the process of step ST49 is executed, the flight imaging support process proceeds to step ST50.
  • step ST50 the flight instruction transmission control section 170 transmits flight instructions to the aircraft 310 via the communication device 12.
  • step ST50 the flight imaging support process proceeds to step ST42.
  • step ST48 the determination in step ST48 is affirmative, and the flight imaging support process proceeds to step ST60 shown in FIG. Transition.
  • step ST60 shown in FIG. 37 the operation mode setting unit 102 sets the operation mode of the base station 10 to the imaging control mode. After the process of step ST60 is executed, the flight imaging support process proceeds to step ST61.
  • step ST61 the hovering instruction transmission control section 172 transmits a hovering instruction to the aircraft 310 via the communication device 12.
  • the flight imaging support process proceeds to step ST62.
  • the hovering instruction is transmitted to the flying object 310 by executing the process of step ST61
  • the process of steps ST92 to ST94 (see FIG. 40) of the flight imaging process is executed by the processor 351 of the flying object 310.
  • a hovering report is transmitted from the aircraft 310 to the base station 10 by .
  • step ST62 the hovering report reception determining unit 174 determines whether or not the hovering report transmitted from the aircraft 310 has been received by the communication device 12. In step ST62, when the hovering report is not received by the communication device 12, the determination is negative, and the determination in step ST62 is performed again. In step ST62, when the hovering report is received by the communication device 12, the determination is affirmative, and the flight imaging support process proceeds to step ST63.
  • step ST63 the third imaging control unit 176 causes the imaging device 30 to capture an imaging scene including the flying object 310. After the process of step ST63 is executed, the flight imaging support process proceeds to step ST64.
  • step ST64 the flying object attitude identification unit 178 performs object recognition processing on the image obtained by the image pickup device 30, and based on the positions of the plurality of propellers 341 captured in the image. to identify the attitude of the aircraft 310 .
  • the flight imaging support process proceeds to step ST65.
  • step ST65 the attitude correction instruction generating section 180 generates an attitude correction instruction for the flying object 310 based on the attitude of the flying object 310 specified at step ST64.
  • step ST66 the flight imaging support process proceeds to step ST66.
  • step ST66 the attitude correction instruction transmission control section 182 transmits an attitude correction instruction to the aircraft 310 via the communication device 12.
  • the flight imaging support process proceeds to step ST70.
  • the attitude correction instruction is transmitted to the flying object 310 by executing the process of step ST66
  • the process of steps ST100 to ST102 (see FIG. 41) of the flight imaging process is executed by the processor 351 of the flying object 310, As a result, an attitude correction report is transmitted from the aircraft 310 to the base station 10 .
  • step ST70 the attitude correction report reception determination unit 184 determines whether or not the communication device 12 has received the attitude correction report transmitted from the aircraft 310. In step ST70, if the attitude correction report has not been received by the communication device 12, the determination is negative, and the determination in step ST70 is performed again. In step ST70, when the attitude correction report is received by the communication device 12, the determination is affirmative, and the flight imaging support process proceeds to step ST71.
  • the zoom magnification determination unit 186 determines whether the zoom magnification stored in the storage 52 at step ST26 or step ST29 is the first zoom magnification or the second zoom magnification. In step ST71, when the zoom magnification stored in the storage 52 is the first zoom magnification, the flight imaging support processing proceeds to step ST72. In step ST71, when the zoom magnification stored in the storage 52 is the second zoom magnification, the flight imaging support processing proceeds to step ST73.
  • step ST72 the first angle-of-view setting instruction transmission control unit 188 transmits a first angle-of-view setting instruction corresponding to the first zoom magnification to the aircraft 310 via the communication device 12.
  • step ST72 the flight imaging support process proceeds to step ST80.
  • the process of steps ST103 to ST105 (see FIG. 41) of the flight imaging process is executed by the processor 351 of the flying object 310.
  • a field angle setting report is transmitted from the aircraft 310 to the base station 10 by this.
  • step ST73 the distance derivation unit 190 derives the distance between the inspection target surface 4G and the target imaging position 8A based on the distance information stored in the storage 52 at step ST15. After the process of step ST73 is executed, the flight imaging support process proceeds to step ST74.
  • step ST74 the second angle-of-view setting instruction generation unit 192 adjusts the second zoom magnification to the zoom magnification that makes the pixel resolution of the imaging device 330 the default value described above, based on the distance derived in step ST73. Then, the second angle-of-view setting instruction generator 192 generates a second angle-of-view setting instruction corresponding to the second zoom magnification adjusted based on the distance derived in step ST73. After the process of step ST74 is executed, the flight imaging support process proceeds to step ST75.
  • step ST75 the second angle-of-view setting instruction transmission control unit 194 performs control to transmit the second angle-of-view setting instruction generated in step ST74 to the aircraft 310 via the communication device 12.
  • step ST80 the flight imaging support process proceeds to step ST80.
  • the process of steps ST103 to ST105 (see FIG. 41) of the flight imaging process is executed by the processor 351 of the flying object 310.
  • a field angle setting report is transmitted from the aircraft 310 to the base station 10 by this.
  • the flying object 310 transmits the angle of view setting report to the base station 10 by executing the process of step ST72, and the flying object 310 transmits the image to the base station 10 by executing the process of step ST75.
  • An angle setting report is sent. Therefore, in step ST80, the view angle setting report reception determination unit 196 determines whether or not the view angle setting report transmitted from the flying object 310 has been received by the communication device 12 or not. In step ST80, if the communication device 12 has not received the view angle setting report, the determination is negative, and the determination in step ST80 is performed again. In step ST80, if the communication device 12 receives the view angle setting report, the determination is affirmative, and the flight imaging support process proceeds to step ST81.
  • step ST81 the imaging instruction transmission control unit 198 transmits an imaging instruction to the flying object 310 via the communication device 12.
  • the flight imaging support process proceeds to step ST82.
  • the imaging instruction is transmitted to the flying object 310 by executing the process of step ST81
  • the process of steps ST110 to ST113 (see FIG. 42) of the flight imaging process is executed by the processor 351 of the flying object 310.
  • An imaging report is transmitted from the aircraft 310 to the base station 10 by .
  • step ST82 the imaging report reception determination unit 200 determines whether or not the imaging report transmitted from the flying object 310 has been received by the communication device 12. In step ST82, if the imaging report has not been received by the communication device 12, the determination is negative, and the determination in step ST82 is performed again. In step ST82, when the imaging report is received by the communication device 12, the determination is affirmative, and the flight imaging support process proceeds to step ST83.
  • step ST83 the termination determination unit 202 determines whether or not the conditions for terminating the flight imaging support process are satisfied.
  • An example of a condition for ending the flight imaging support process is that the number of imaging reports received in step ST82 (that is, the number of times the determination in step ST82 is affirmative) reaches the number of imaging positions 8A.
  • step ST83 if the condition for ending the flight imaging support process is not satisfied, the determination is negative, and the flight imaging support process proceeds to step ST42. Then, a plurality of images are acquired by repeatedly executing the flight imaging support process described above.
  • step ST83 if the condition for ending the flight imaging support process is satisfied, the determination is affirmative, and the flight imaging support process proceeds to step ST84.
  • step ST84 the termination instruction transmission control section 204 transmits an termination instruction to the aircraft 310 via the communication device 12. After the process of step ST84 is executed, the flight imaging support process ends.
  • FIG. 40 An example of the flow of flight imaging processing performed by the processor 351 of the flying object 310 will be described with reference to FIGS. 40 to 42.
  • FIG. 40 An example of the flow of flight imaging processing performed by the processor 351 of the flying object 310 will be described with reference to FIGS. 40 to 42.
  • step ST90 the flight instruction reception determination unit 402 determines whether or not the communication device 312 has received a flight instruction. In step ST90, if no flight instruction has been received by communication device 312, the determination is negative, and the flight imaging process proceeds to step ST92. In step ST90, if the communication device 312 receives a flight instruction, the determination is affirmative, and the flight imaging process proceeds to step ST91.
  • step ST91 the flight control section 404 controls the flight device 340 according to the flight instruction. After the process of step ST91 is executed, the flight imaging process proceeds to step ST92.
  • step ST92 the hovering instruction reception determination unit 406 determines whether or not the communication device 312 has received a hovering instruction. In step ST92, if no hovering instruction has been received by communication device 312, the determination is negative, and the flight imaging process proceeds to step ST100. In step ST92, if the hovering instruction is received by the communication device 312, the determination is affirmative, and the flight imaging process proceeds to step ST93.
  • step ST93 the hovering control section 408 causes the flying object 310 to hover.
  • step ST93 the flight imaging process proceeds to step ST94.
  • step ST94 the hovering report transmission control section 410 transmits the hovering report to the base station 10 via the communication device 312. After the process of step ST94 is executed, the flight imaging process proceeds to step ST100.
  • step ST100 the attitude correction instruction reception determination unit 412 determines whether or not the communication device 312 has received an attitude correction instruction. In step ST100, if the attitude correction instruction has not been received by the communication device 312, the determination is negative, and the flight imaging process proceeds to step ST103. In step ST100, if the communication device 312 receives an attitude correction instruction, the determination is affirmative, and the flight imaging process proceeds to step ST101.
  • step ST101 the attitude correction control section 414 corrects the attitude of the flying object 310 according to the attitude correction instruction.
  • step ST101 the attitude correction control section 414 corrects the attitude of the flying object 310 according to the attitude correction instruction.
  • step ST102 the attitude correction report transmission control section 416 transmits the attitude correction report to the base station 10 via the communication device 312 .
  • step ST102 the flight imaging process proceeds to step ST103.
  • step ST103 the angle-of-view setting instruction reception determination unit 418 determines whether or not the communication device 312 has received an angle-of-view setting instruction. In step ST103, if the communication device 312 has not received an angle-of-view setting instruction, the determination is negative, and the flight imaging process proceeds to step ST110. In step ST103, when the communication device 312 receives the angle of view setting instruction, the determination is affirmative, and the flight imaging process proceeds to step ST104.
  • step ST104 the field angle control unit 420 sets the field angle of the imaging device 330 to the field angle corresponding to the field angle setting instruction. After the process of step ST104 is executed, the flight imaging process proceeds to step ST105.
  • step ST105 the view angle setting report transmission control section 422 transmits the view angle setting report to the base station 10 via the communication device 312. After the process of step ST105 is executed, the flight imaging process proceeds to step ST110.
  • step ST110 the imaging instruction reception determination unit 424 determines whether or not the communication device 312 has received an imaging instruction. In step ST110, if the imaging instruction has not been received by communication device 312, the determination is negative, and the flight imaging process proceeds to step ST114. In step ST110, if the communication device 312 has received the imaging instruction, the determination is affirmative, and the flight imaging process proceeds to step ST111.
  • step ST111 the imaging control unit 426 causes the imaging device 330 to image the inspection target surface 4G. After the process of step ST111 is executed, the flight imaging process proceeds to step ST112.
  • step ST112 the image information storage control unit 118 causes the image memory 314 to store the image obtained by being imaged by the imaging device 330. After the process of step ST112 is executed, the flight imaging process proceeds to step ST113.
  • step ST113 the imaging report transmission control section 430 transmits the imaging report to the base station 10 via the communication device 312. After the process of step ST113 is executed, the flight imaging process proceeds to step ST114.
  • step ST114 the termination instruction reception determination unit 432 determines whether or not the communication device 312 has received the termination instruction. In step ST114, if the communication device 312 has not received the end instruction, the determination is negative, and the flight imaging process proceeds to step ST90. In step ST114, when the communication device 312 receives the end instruction, the determination is affirmative, and the flight imaging process proceeds to step ST115.
  • the termination control section 434 terminates the flight of the aircraft 310.
  • the control for ending the flight by the end control unit 434 includes, for example, control for landing the flying object 310, control for returning the flying object 310 to the position where the flying object 310 started the flight imaging process, and/or the flying object 310. Examples include control for switching to operation by a pilot (not shown). After the process of step ST115 is executed, the flight imaging process ends.
  • control method described as the action of the imaging system S described above is an example of the "control method” according to the technology of the present disclosure.
  • the processor 51 rotates the distance measuring device 40 with respect to the rotation driving device 20 to which the distance measuring device 40 is attached, and rotates the wall surface 4 with respect to the distance measuring device 40 .
  • the distance between the wall surface 4 and the distance measuring device 40 is measured at a plurality of distance measuring points.
  • the processor 51 sets a flight route 8 along which the aircraft 310 flies along the wall surface 4 based on the distances measured at each distance measurement point.
  • the processor 51 controls the flying object 310 to fly along the flight route 8 and controls the imaging device 330 mounted on the flying object 310 to image a plurality of areas to be imaged on the wall surface 4. . Therefore, for example, without using a satellite positioning system, the flying object 310 can fly along the wall surface 4 and a plurality of areas to be imaged on the wall surface 4 can be imaged by the imaging device 330 .
  • the processor 51 causes the flying object 310 to fly along the flight route 8 and causes the imaging device 330 mounted on the flying object 310 to image a plurality of areas to be imaged on the wall surface 4 .
  • control is performed to keep the pixel resolution of the imaging device 330 constant. Therefore, even if the wall surface 4 has a concave portion 4F, for example, the resolution of the image can be kept constant.
  • the processor 51 adjusts the rotation angle of the rotary drive device 20 to a rotation angle at which the flying object 310 is included in the ranging range 41 of the ranging device 40, The distance between the distance device 40 is measured. Then, the processor 51 controls the flying object 310 to fly along the flight route 8 based on the rotation angle of the rotary drive device 20 and the distance between the flying object 310 and the distance measuring device 40 . Therefore, for example, the flying object 310 can fly over a wider range than when the ranging range 41 of the ranging device 40 is fixed.
  • the processor 51 also calculates the absolute coordinates of the rotary drive device 20, the rotation angle of the rotary drive device 20, the angle of the laser beam emitted from the rangefinder 40 toward the aircraft 310, and the coordinates of the aircraft 310 and the rangefinder 40.
  • the absolute coordinates of the aircraft 310 are derived based on the distance between .
  • the flying object 310 is controlled to fly the flight route 8 . Therefore, for example, the flying object 310 can be flown along the wall surface 4 based on the absolute coordinates of the flying object 310 without using a satellite positioning system.
  • the processor 51 determines the rotation angle of the rotation drive device 20 based on the image obtained by imaging the flying object 310 with the imaging device 30 . control to adjust the rotation angle. Therefore, for example, the distance measuring range 41 of the distance measuring device 40 can be moved to follow the flying object 310 .
  • the processor 51 performs control to adjust the rotation angle of the rotary drive device 20 to an angle at which the flying object 310 is positioned at the center of the angle of view of the imaging device 30 . Therefore, for example, compared to the case where the rotation angle of the rotary drive device 20 is adjusted to an angle in which the flying object 310 is positioned outside the center of the angle of view of the imaging device 30, the flying object 310 does not move. Also, it is possible to prevent the flying object 310 from deviating from the angle of view of the imaging device 30 .
  • the flying object 310 includes a plurality of propellers 341 classified in different manners. Then, the processor 51 controls the attitude of the flying object 310 based on the positions of the propellers 341 captured in the images captured by the imaging device 30 . Therefore, for example, the attitude of the flying object 310 can be controlled with higher accuracy than when the plurality of propellers 341 are not classified in different manners.
  • the plurality of propellers 341 are classified by different colors. Therefore, for example, the attitude of the flying object 310 can be identified by a simple configuration in which the propellers 341 have different colors.
  • the flying object 310 acquires a plurality of images each time it reaches each of the plurality of imaging positions 8A set on the flight route 8. Therefore, for example, the condition of the wall surface 4 can be inspected by analyzing a plurality of images with the image analysis device 2 .
  • the plurality of imaging positions 8A are set at positions where the images acquired at adjacent imaging positions 8A among the plurality of imaging positions 8A partially overlap. Therefore, for example, the image analysis device 2 can recognize that the images are adjacent images based on the amount of overlap between the images.
  • the processor 51 operates at least one of the zoom lens 335C and the focus lens 335B of the imaging device 330 to keep the pixel resolution of the imaging device 330 constant. keep in control. Therefore, for example, even when the flying object 310 flies over the concave portion 4F, the pixel resolution of the imaging device 330 can be kept constant.
  • the flying object 310 may include a first member 360A, a second member 360B, a third member 360C, and a fourth member 360D.
  • the first member 360A is arranged on the front right side of the aircraft body 320
  • the second member 360B is arranged on the front left side of the aircraft body 320
  • the third member 360C is arranged on the aircraft body 320.
  • the fourth member 360D is arranged on the rear left side of the aircraft main body 320. As shown in FIG.
  • the first member 360A and the third member 360C are arranged on the right side with respect to the imaging device 330, and the second member 360B and the fourth member 360D are arranged on the left side with respect to the imaging device 330.
  • the first member 360A is arranged at a line-symmetrical position with respect to the second member 360B centering on the optical axis OA2 of the image pickup device 330 in plan view
  • the third member 360C is arranged at a line-symmetrical position with respect to the optical axis OA2 of the image pickup device 330 in plan view. centered on the fourth member 360D.
  • the first member 360A, the second member 360B, the third member 360C, and the fourth member 360D are examples of the "plurality of members" according to the technology of the present disclosure.
  • the first member 360A, the second member 360B, the third member 360C, and the fourth member 360D are classified with different colors as examples of different modes.
  • dots attached to the first member 360A, the second member 360B, the third member 360C, and the fourth member 360D express the color of each member.
  • the color of the first member 360A is the same as the color of the second member 360B, and the color of the third member 360C is the same as the color of the fourth member 360D.
  • the first color set for the first member 360A and the second member 360B is different from the second color set for the third member 360C and the fourth member 360D.
  • the first color and the second color may each be chromatic or achromatic.
  • the first color and the second color can be identified based on an image obtained by the processor 51 (see FIG. 4) of the base station 10 (described later) being imaged by the imaging device 30. Any color may be used as long as it is a color.
  • the first color is set for the first member 360A and the second member 360B
  • the second color is set for the third member 360C and the fourth member 360D
  • the first color may be set for the first member 360A and the third member 360C
  • the second color may be set for the second member 360B and the fourth member 360D
  • a first color may be set for the first member 360A and the fourth member 360D
  • a second color may be set for the second member 360B and the third member 360C.
  • different colors may be set for the first member 360A, the second member 360B, the third member 360C, and the fourth member 360D.
  • first member 360A, the second member 360B, the third member 360C, and the fourth member 360D may be light emitters that emit light of different colors as examples of different modes. Furthermore, the first member 360A, the second member 360B, the third member 360C, and the fourth member 360D may be light emitters that blink in different blinking patterns as examples of different modes.
  • the processor 51 instead of the distance determination unit 136, the processor 51 performs image recognition processing on the image information stored in the storage 52, and the image represented by the image information corresponds to the concave portion 4F. By determining whether an image is included, it may be determined whether or not the surface 4G to be inspected has the concave portion 4F.
  • the processing by the first zoom magnification determination unit 138, the first zoom magnification storage control unit 140, and the first flight route setting unit 142 is executed, and the inspection target surface 4G is does not have the concave portion 4F, the processing by the second zoom magnification determination section 144, the second zoom magnification storage control section 146, and the second flight route setting section 148 may be executed. Even in this case, the image resolution can be kept constant.
  • the processor 51 determines whether or not the surface 4G to be inspected has the recessed portion 4F, and when it is determined that the surface 4G to be inspected has the recessed portion 4F, the area of the opening 4F1 of the recessed portion 4F is predetermined. You may determine whether it is smaller than an area.
  • the predetermined opening area is set, for example, to be less than the area that allows the flying object 310 to enter the recess 4F.
  • the processor 51 may set the flight route 8 along the inspection target surface 4G.
  • the processor 51 determines that the surface to be inspected 4G has the concave portion 4F and the area of the opening portion 4F1 of the concave portion 4F is equal to or larger than the predetermined area, the processor 51 performs flight along the inner surface of the concave portion 4F along the copying surface 6.
  • Route 8 may be set.
  • the processor 51 determines that the surface to be inspected 4G has the recessed portion 4F and the area of the opening 4F1 of the recessed portion 4F is smaller than the predetermined area, the processor 51 determines that the smooth surface facing the surface to be inspected 4G having the recessed portion 4F has the recessed portion 4F.
  • a flight route 8 may be set on the surface 7 (that is, a smooth virtual surface facing the inspection target surface 4G). Even in this case, the image resolution can be kept constant.
  • the inspection object 3 has the concave portion 4F, but may have a convex portion instead of the concave portion 4F.
  • the processor 51 causes the flying object 310 to fly along the flight route 8 and causes the imaging device 330 mounted on the flying object 310 to image the inspection target surface 4G, thereby acquiring a plurality of images.
  • control may be performed to keep the pixel resolution of the imaging device 330 constant.
  • the imaging system S includes a first base station 10A and a second base station 10B as an example of a plurality of base stations.
  • the imaging system S includes a controller 60 common to the first base station 10A and the second base station 10B.
  • Controller 60 comprises accepting device 14 , display 16 and computer 150 .
  • a computer 150 includes a processor 51, a storage 52, and a RAM 53, and the processor 51, storage 52, RAM 53, reception device 14, and display 16 are connected to a bus, as in the first embodiment.
  • Each base station 10 includes a rotation drive device 20 , an imaging device 30 and a distance measuring device 40 .
  • the rotary drive device 20 , imaging device 30 , and distance measuring device 40 are electrically connected to the controller 60 .
  • the configurations of the rotary drive device 20, the imaging device 30, and the distance measuring device 40 are the same as in the first embodiment.
  • the first base station 10A and the second base station 10B can image the wall surface 4 of the inspection object 3 with the imaging device 30, and measure the distance between the wall surface 4 and the distance measuring device 40 with the distance measuring device 40. Installed in a measurable position. For example, if the inspection object 3 is a bridge across a river, the first base station 10A is installed on one side of the river, and the second base station 10B is installed on the other side of the river.
  • the first base station 10A and the second base station 10B are installed at positions where the ranging areas of the respective ranging devices 40 partially overlap.
  • the example which the rangefinder 40 irradiates a laser beam toward diagonally upward as an example is demonstrated.
  • the rotary drive device 20, the imaging device 30, and the distance measuring device 40 of the first base station 10A are the “first rotary driving device”, the “first imaging device”, and the “first distance measuring device” according to the technology of the present disclosure.
  • the rotary drive device 20, the imaging device 30, and the distance measuring device 40 of the second base station 10B are the "second rotary driving device”, the “second imaging device”, and the “second distance measuring device” according to the technology of the present disclosure. is an example of
  • the flight route setting processing unit 104 includes a first acceptance determination unit 112, a first rotation control unit 114, a first imaging control unit 116, an image information storage control unit 118, a first ranging control unit 120, distance information storage control unit 122, rotation position determination unit 124, rotation stop control unit 126, image display control unit 128, second acceptance determination unit 130, copying surface setting unit 132, smooth surface setting unit 134, distance determination unit 136, a first zoom magnification determination unit 138, a first zoom magnification storage control unit 140, a first flight route setting unit 142, a second zoom magnification determination unit 144, a second zoom magnification storage control unit 146, and a second flight route setting
  • the unit 148 it has a calibration information derivation unit 212 and a calibration information storage control unit 214 .
  • the flight control processing unit 106 includes a third reception determination unit 152, a second imaging control unit 154, an aircraft position derivation unit 156, a position deviation determination unit 158, a second rotation control unit 160,
  • the first aircraft determination section 216 is provided.
  • the imaging control processing unit 108 includes a hovering instruction transmission control unit 172, a hovering report reception determination unit 174, a third imaging control unit 176, an aircraft attitude identification unit 178, and an attitude correction instruction generation unit 180.
  • attitude correction instruction transmission control unit 182 attitude correction report reception determination unit 184
  • zoom magnification determination unit 186 first angle of view setting instruction transmission control unit 188
  • distance derivation unit 190 second angle of view setting instruction generation unit 192
  • angle-of-view setting report reception determination unit 196 the imaging instruction transmission control unit 198
  • imaging report reception determination unit 200 the end determination unit 202
  • end instruction transmission control unit 204 It has two flying object determination units 218 .
  • worker 5 gives a measurement start instruction to reception device 14 .
  • the first reception determination unit 112 determines whether or not the measurement start instruction has been received by the reception device 14 .
  • the first rotation control unit 114 moves the pedestal 27 to the first rotation position with respect to the rotation drive device 20 of each base station 10 . to the second rotation position.
  • An example in which the first rotation control unit 114 synchronously rotates the pedestals 27 of the base stations 10 will be described below as an example.
  • the first imaging control unit 116 controls the imaging device 30 of each base station 10 to image the wall surface 4 .
  • the image information storage control unit 118 detects images obtained by imaging by the imaging device 30 of each base station 10 using a rotation detector (not shown) provided on the pan/tilt mechanism 26 and/or the pedestal 27. Image information is generated by associating the detected rotational position of the pedestal 27 and the image information is stored in the storage 52 .
  • the first ranging control unit 120 controls the ranging device 40 of each base station 10 to scan the wall surface 4 with laser light. In one scan of the distance measuring device 40 of each base station 10, the distance between the wall surface 4 and the distance measuring device 40 is measured at a plurality of distance measuring points on a part of the wall surface 4 in the horizontal direction.
  • the measurement of the first base station 10A A ranging point measured by the ranging device 40 is called a first ranging point, and a ranging point measured by the ranging device 40 of the second base station 10B is called a second ranging point.
  • the first ranging point is an example of the "first ranging point” according to the technology of the present disclosure
  • the second ranging point is an example of the "first ranging point” according to the technology of the present disclosure
  • the distance between the wall surface 4 and the range finder 40 measured by the range finder 40 of the first base station 10A is an example of the "first distance” according to the technology of the present disclosure
  • the distance of the second base station 10B The distance between the wall surface 4 and the distance measuring device 40 measured by the distance measuring device 40 is an example of the "second distance” according to the technology of the present disclosure.
  • the distance information storage control unit 122 stores the distance measured for each distance measurement point in each base station 10 by the pedestal detected by the rotation detector (not shown) provided on the pan/tilt mechanism 26 and/or the pedestal 27. 27 and the rotation angle of the scanner mirror 47 detected by an angle detector (not shown) provided on the scanner mirror 47 to generate distance information and store the distance information in the storage 52.
  • the rotational position determination unit 124 determines whether or not the rotational position of the pedestal 27 of each base station 10 has reached the second rotational position.
  • the rotational position determination unit 124 compares the rotational position of the pedestal 27 detected by, for example, the pan-tilt mechanism 26 and/or a rotation detector (not shown) provided on the pedestal 27 with the position of the second rotational position. determines whether or not the rotational position of the pedestal 27 has reached the second rotational position.
  • the rotation stop control unit 126 causes each rotary drive device 20 to rotate the pedestal 27. Control to stop.
  • the imaging device 30 captures images of the wall surface 4 a plurality of times
  • the distance measuring device 40 captures images of the wall surface 4 . Image information and distance information corresponding to the wall surface 4 are obtained by scanning the wall surface 4 in a plurality of times.
  • the image display control unit 128 displays an image (that is, an image showing the wall surface 4) on the display 16 based on the image information stored in the storage 52. Control the display.
  • the operator 5 determines the inspection target surface 4G to be inspected by the flying object 310. Then, the worker 5 provides the reception device 14 with inspection target surface designation information indicating that the inspection target surface 4G is designated.
  • inspection target surface designation information indicating that the inspection target surface 4G is designated.
  • the operator 5 determines a plurality of positions on the wall surface 4 from areas where the range-finding areas of the range-finding devices 40 overlap. Then, position specifying information for specifying a plurality of positions is given to the receiving device 14 .
  • An example in which the points A and B of the wall surface 4 are determined as a plurality of positions on the wall surface 4 as shown in FIG. 50 will be described below. Points A and B are horizontally and vertically separated from each other.
  • the second reception determination unit 130 determines whether or not the inspection target surface designation information and the position designation information have been received by the reception device 14 .
  • the calibration information derivation unit 212 derives calibration information based on the position designation information and the distance information.
  • the calibration information is the distance measured by the distance measuring device 40 of the second base station 10B (that is, the distance between the wall surface 4 and the second base station 10B) and the This is information for conversion into a distance based on the position of the distance measuring device 40 .
  • the calibration information is for converting the position of the flying object 310 measured by the rangefinder 40 of the second base station 10B into a position based on the position of the rangefinder 40 of the first base station 10A.
  • Information Specifically, the calibration information derivation unit 212 derives the calibration information in the following procedure.
  • the calibration information derivation unit 212 calculates the length La1 of the side A1 based on the distance information.
  • a side A1 is a side connecting the point A and the point C1 of the first base station 10A.
  • calibration information derivation section 212 calculates angle ⁇ ac1 based on the distance information.
  • the angle ⁇ ac1 is the angle formed by the side A1 and the side C.
  • a side C is a side connecting a point C1 of the first base station 10A and a point C2 of the second base station 10B.
  • the calibration information derivation unit 212 calculates the length Lb1 of the side B1 based on the distance information.
  • a side B1 is a side connecting a point C1 and a point B indicating the position where the first base station 10A is installed.
  • the calibration information derivation unit 212 calculates the angle ⁇ bc1 based on the distance information.
  • the angle ⁇ bc1 is the angle formed by the side B1 and the side C.
  • the calibration information derivation unit 212 calculates the angle ⁇ ab1 based on the following formula (1).
  • the angle ⁇ ab1 is the angle formed by the side A1 and the side B1.
  • the calibration information derivation unit 212 calculates the length La2 of the side A2 based on the distance information.
  • Side A2 is a side connecting point A and point C2 indicating the position where the second base station 10B is installed.
  • the calibration information deriving section 212 calculates the angle ⁇ ac2 based on the distance information.
  • the angle ⁇ ac2 is the angle formed by the side A2 and the side C.
  • the calibration information derivation unit 212 calculates the length Lb2 of the side B2 based on the distance information.
  • a side B2 is a side connecting the point C2 and the point B of the second base station 10B.
  • the calibration information deriving section 212 calculates the angle ⁇ bc2 based on the distance information.
  • the angle ⁇ bc2 is the angle formed by the side B2 and the side C.
  • the calibration information derivation unit 212 calculates the angle ⁇ ab2 based on the following formula (2).
  • the angle ⁇ ab2 is the angle formed by the side A2 and the side B2.
  • the calibration information derivation unit 212 calculates the angle ⁇ 1 based on the following formula (3) based on the law of cosines.
  • the angle ⁇ 1 is the angle between the side A1 and the side AB.
  • a side AB is a side connecting the points A and B.
  • the calibration information derivation unit 212 calculates the angle ⁇ 2 based on the following formula (4) based on the law of cosines.
  • the angle ⁇ 2 is the angle between the side A2 and the side AB.
  • the calibration information derivation unit 212 calculates the angle ⁇ based on the following formula (5).
  • the calibration information derivation unit 212 calculates the length Lc of the side C based on the following formula (6) based on the law of cosines. Also, the calibration information derivation unit 212 derives the coordinates of the side C as an angle reference.
  • the length Lc calculated by Equation (6) (that is, the first base station 10A and the second base station 10B ), the length Ld2 and the angle ⁇ 2 of the side D2 measured at the second base station 10B can be obtained from the first base station 10A based on the following equations (7) and (8): It can be converted into the length Ld1 and the angle ⁇ 1 of the side D1 which are measured in a pseudo manner.
  • the side D1 is the side connecting the position D and the point C1 of the first base station 10A
  • the side D2 is the side connecting the position D and the point C2 of the second base station 10B.
  • Angle ⁇ 1 and angle ⁇ 2 are angles with side C as a reference.
  • the angle ⁇ 1 is the angle formed by the side D1 and the side C
  • the angle ⁇ 2 is the angle formed by the side D2 and the side C.
  • the distance measured by the distance measuring device 40 of the second base station 10B is converted into a distance based on the position of the first base station 10A by the following formula (7).
  • the position of the first base station 10A is synonymous with the position of the ranging device 40 of the first base station 10A.
  • the rotation angle of the rotary drive device 20 of the second base station 10B is converted into an angle based on the position of the first base station 10A by the following formula (8).
  • the calibration information storage control unit 214 uses the conversion formula obtained by substituting the value of the length Lc calculated by the formula (6) into the following formulas (7) and (8) and the coordinates of the side C as calibration information. Store in the storage 52 .
  • the calibration information stored in the storage 52 is an example of "default first calibration information" and "default second calibration information” according to the technology of the present disclosure.
  • the image display control unit 128 displays an image (that is, an image of the wall surface 4) on the display 16 based on the image information stored in the storage 52. Control to display.
  • the operator 5 determines the inspection target surface 4G based on the image displayed on the display 16. Then, the worker 5 provides the reception device 14 with inspection target surface designation information indicating that the inspection target surface 4G is designated.
  • the second reception determination unit 130 determines whether or not the inspection target surface specification information has been received by the reception device 14 .
  • the copying surface setting unit 132 sets the copying surface 6 based on the inspection surface designation information.
  • the scanning surface 6 is the first scanning surface 6A positioned within the ranging area of the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B. and a second copying surface 6B positioned within the range-finding area of .
  • the scanning plane setting unit 132 sets the second scanning plane 6B based on the calibration information stored in the storage 52, based on the relative coordinates with reference to the position of the first base station 10A.
  • a copy plane 6B is set.
  • the entire copying surface 6 is set based on the relative coordinates based on the position of the first base station 10A.
  • the smooth surface setting unit 134 smoothes the copying surface 6 to set the smooth surface 7 (that is, the smooth virtual surface facing the wall surface 4). Like the copying surface 6, the smooth surface 7 is also set based on relative coordinates with reference to the position of the first base station 10A. The method by which the smooth surface setting unit 134 sets the smooth surface 7 is the same as in the first embodiment.
  • the functions of the magnification storage control unit 146 and the second flight route setting unit 148 are the same as in the first embodiment.
  • the first flight route setting unit 142 or the second flight route setting unit 148 sets a flight route 8 passing through a plurality of imaging positions 8A.
  • the flight route 8 is set by relative coordinates based on the position of the first base station 10A.
  • the flying object 310 is arranged within the imaging range 31 of the imaging device 30 of the first base station 10A.
  • the operator 5 issues a flight start instruction to the reception device 14 when the aircraft 310 is ready to start flying.
  • the third reception determination unit 152 determines whether or not the flight start instruction has been received by the reception device 14 .
  • the second imaging control unit 154 controls the imaging device 30 of each base station 10 to image the imaging scene. .
  • the first flying object determination unit 216 performs object recognition processing on images obtained by imaging by the imaging device 30 of each base station 10, thereby determining the first base station 10A and the second base station 10B. It is determined which of the images obtained by the base station 10 has the flying object 310 as an image. According to the determination result of the first flying object determination unit 216, as will be described later, the position of the flying object 310 of the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B is measured. A range finder 40 is selected.
  • the flying object position deriving unit 156 executes object recognition processing on an image in which the flying object 310 is shown as an image, out of the images obtained by the first base station 10A or the images obtained by the second base station 10B. Thus, the position in the image of the flying object 310 included as an image in the image is derived.
  • the positional deviation determining unit 158 Based on the position of the flying object 310 in the image derived by the flying object position deriving unit 156, the positional deviation determining unit 158 adjusts the central portion of the angle of view of the imaging device 30 of the first base station 10A or the second base station 10B. It is determined whether or not the position of the flying object 310 is displaced with respect to .
  • the second rotation control unit 160 controls the rotation angle in the horizontal direction and/or the vertical direction of the rotary drive device 20. is adjusted to an angle at which the flying object 310 is positioned at the center of the angle of view of the imaging device 30 .
  • the second ranging control unit 162 determines whether the flying object 310 of the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B A range finder 40 is selected to measure the position. That is, the second ranging control unit 162 selects the base station 10A or the second base station 10B from which the first flying object determination unit 216 determines that an image of the flying object 310 is obtained. The range finder 40 of the station 10 is selected as the range finder 40 for measuring the position of the aircraft 310 .
  • the second ranging control unit 162 uses the laser beam to measure the distance measuring device 40 selected from the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B. Control to scan the distance range 41 is performed. In this case, since the flying object 310 is positioned within the ranging range 41 of the selected ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained.
  • the flying object coordinate deriving unit 164 determines that the first flying object determining unit 216 has obtained an image of the flying object 310 from the first base station 10A and the second base station 10B. , the rotation angle of the rotation drive device 20, the angle of the laser beam irradiated from the rangefinder 40 toward the aircraft 310, and the distance between the aircraft 310 and the rangefinder 40, each base station Derive the relative coordinates of the vehicle 310 with respect to the 10 positions.
  • the flying object coordinate derivation unit 164 calculates the second base station based on the calibration information stored in the storage 52.
  • the relative coordinates of the aircraft 310 based on the position of 10B are converted into relative coordinates based on the position of the first base station 10A. That is, the position of the aircraft 310 measured by the rangefinder 40 of the second base station 10B is converted into a position based on the position of the first base station 10A.
  • the imaging position arrival determination unit 166 determines the coordinates of the flying object 310 derived by the flying object coordinates deriving unit 164, and the imaging position 8A closest to the flying object 310 among the plurality of imaging positions 8A (hereinafter referred to as the target imaging position 8A). ), it is determined whether or not the flying object 310 has reached the target imaging position 8A. Both the coordinates of the aircraft 310 and the coordinates of the target imaging position 8A are relative coordinates based on the position of the first base station 10A.
  • the flight instruction generating unit 168 combines the coordinates of the flying object 310 derived by the flying object coordinates deriving unit 164 with the target.
  • a flight instruction for the flying object 310 is generated based on the difference between the coordinates of the imaging position 8A.
  • the flight instruction transmission control unit 170 controls transmission of flight instructions to the aircraft 310 via the communication device 12 .
  • the flying object 310 flies toward the target imaging position 8A according to the flight instruction.
  • the hovering instruction transmission control unit 172 controls the communication device 12 to control to transmit a hovering instruction to the flying object 310.
  • the hovering report reception determination unit 174 determines whether the communication device 12 has received a hovering report transmitted from the flying object 310 as the flying object 310 hovers.
  • the third imaging control unit 176 controls the imaging device 30 of each base station 10 to capture the imaging scene.
  • the second flying object determination unit 218 performs object recognition processing on images obtained by imaging by the imaging device 30 of each base station 10, thereby determining the first base station 10A and the second base station 10B. It is determined which of the images obtained by the base station 10 has the flying object 310 as an image.
  • the flying object attitude identification unit 178 executes object recognition processing on an image in which the flying object 310 is shown as an image among the images obtained by the first base station 10A and the images obtained by the second base station 10B.
  • the attitude of the flying object 310 is specified based on the positions of the propellers 341 captured in the image.
  • the attitude correction instruction generating unit 180 generates an attitude correction instruction for the flying object 310 based on the attitude of the flying object 310 identified by the flying object attitude identifying unit 178 .
  • the attitude correction instruction transmission control unit 182 performs control for transmitting an attitude correction instruction to the aircraft 310 via the communication device 12 . As a result, the attitude of the flying object 310 is corrected.
  • the attitude correction report reception determination unit 184 the zoom magnification determination unit 186, the first angle of view setting instruction transmission control unit 188, the distance derivation unit 190, the second angle of view setting instruction generation unit 192, and the second angle of view
  • the functions of the setting instruction transmission control unit 194, the view angle setting report reception determination unit 196, the imaging instruction transmission control unit 198, the imaging report reception determination unit 200, the end determination unit 202, and the end instruction transmission control unit 204 are the same as those in the first embodiment. is similar to
  • FIG. 55 An example of the flow of flight imaging support processing performed by the processor 51 of the controller 60 in the imaging system S according to the second embodiment will be described with reference to FIGS. 55 to 59.
  • FIG. 55 An example of the flow of flight imaging support processing performed by the processor 51 of the controller 60 in the imaging system S according to the second embodiment will be described with reference to FIGS. 55 to 59.
  • step ST210 the operation mode setting section 102 sets the operation mode of the base station 10 to the flight route setting mode. After the process of step ST210 is executed, the flight imaging support process proceeds to step ST211.
  • step ST211 the first reception determination section 112 determines whether or not the reception device 14 has received the measurement start instruction. In step ST211, if the measurement start instruction has not been received by the receiving device 14, the determination is negative, and the determination in step ST211 is performed again. In step ST211, when the measurement start instruction is accepted by the accepting device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST212.
  • step ST212 the first rotation control section 114 rotates the pedestal 27 from the first rotation position toward the second rotation position by controlling the rotary drive device 20 of each base station 10 based on the measurement start instruction. After the process of step ST212 is executed, the flight imaging support process proceeds to step ST213.
  • step ST213 the first imaging control unit 116 causes the imaging device 30 of each base station 10 to image the wall surface 4.
  • step ST213 the flight imaging support process proceeds to step ST214.
  • step ST214 the image information storage control unit 118 causes the storage 52 to store the image information generated by associating the rotation position detected by the rotation detector with the image obtained in each base station 10 in step ST213. After the process of step ST214 is executed, the flight imaging support process proceeds to step ST215.
  • step ST215 the first ranging control unit 120 causes the ranging device 40 of each base station 10 to scan the wall surface 4. After the process of step ST215 is executed, the flight imaging support process proceeds to step ST216.
  • step ST216 distance information storage control section 122 associates the rotational position detected by the rotation detector and the rotational angle detected by the angle detector with the distance measured at each base station 10 in step ST215.
  • the distance information generated by is stored in the storage 52 .
  • step ST217 the rotational position determination unit 124 determines whether or not the rotational position of the pedestal 27 of each base station 10 has reached the second rotational position. In step ST217, if the rotational position of the pedestal 27 of each base station 10 has not reached the second rotational position, the determination is negative, and the flight imaging support process proceeds to step ST213.
  • step ST213 and ST214 are repeatedly executed, so that a plurality of areas to be imaged on the wall surface 4 are continuously captured. imaged. Image information corresponding to each imaged area is stored in the storage 52 . Further, by repeatedly executing the above-described steps ST215 and ST216 until the rotational position of the pedestal 27 of each base station 10 reaches the second rotational position, the plurality of ranging areas of the wall surface 4 are respectively It is continuously scanned by laser light. Distance information corresponding to each ranging area is stored in the storage 52 . In step ST217, when the rotational position of the pedestal 27 of each base station 10 reaches the second rotational position, the determination is affirmative, and the flight imaging support process proceeds to step ST218.
  • step ST218 the rotation stop control section 126 stops the rotation of the pedestal 27 by stopping the rotation of the rotary drive device 20 of each base station 10. After the process of step ST218 is executed, the flight imaging support process proceeds to step ST220.
  • step ST220 the image display control unit 128 causes the display 16 to display an image based on the image information stored in the storage 52.
  • the wall surface 4 is represented as an image in the image.
  • step ST221 the second reception determination unit 130 determines whether or not the inspection surface designation information and the position designation information provided by the worker 5 have been received by the reception device 14. In step ST221, if the inspection target surface designation information and the position designation information have not been received by the reception device 14, the determination is negative, and the determination in step ST221 is performed again. In step ST221, if the inspection target surface designation information and the position designation information are received by the reception device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST221A.
  • step ST221A the calibration information derivation unit 212 derives calibration information based on the position designation information and the distance information. After the process of step ST221A is executed, the flight imaging support process proceeds to step ST221B.
  • step ST221B the calibration information storage control section 214 stores the calibration information in the storage 52. After the process of step ST221B is executed, the flight imaging support process proceeds to step ST222.
  • step ST222 the copying plane setting unit 132 sets the copying plane 6 that copies the inspection plane 4G based on the inspection plane designation information and the calibration information. After the process of step ST222 is executed, the flight imaging support process proceeds to step ST223.
  • step ST223 the smooth surface setting section 134 sets the smooth surface 7 by smoothing the copy surface 6. After the process of step ST223 is executed, the flight imaging support process proceeds to step ST224.
  • step ST224 the distance determination unit 136 determines whether or not the distance between the inspection target surface 4G and the smooth surface 7 is constant based on the distance information stored in the storage 52. In step ST224, if the distance between the inspection target surface 4G and the smooth surface 7 is constant, the determination is affirmative, and the flight imaging support process proceeds to step ST225. In step ST224, if the distance between the surface 4G to be inspected and the smooth surface 7 is not constant, the determination is negative, and the flight imaging support process proceeds to step ST228.
  • the first zoom magnification determination unit 138 determines the zoom magnification of the imaging device 330 of the flying object 310 to be the first zoom magnification.
  • the first zoom magnification is a zoom magnification at which the pixel resolution of the imaging device 330 is a default value.
  • step ST226 the first zoom magnification storage control section 140 stores the first zoom magnification determined by the first zoom magnification determination section 138 in the storage 52. After the process of step ST226 is executed, the flight imaging support process proceeds to step ST227.
  • step ST227 the first flight route setting section 142 sets a plurality of imaging positions 8A on the smooth surface 7 based on the first zoom magnification determined by the first zoom magnification determining section 138, thereby obtaining a plurality of A flight route 8 passing through the imaging position 8A is set.
  • the first flight route setting unit 142 images the inspection target surface 4G at the first zoom magnification determined by the first zoom magnification determining unit 138
  • the first flight route setting unit 142 selects adjacent imaging positions 8A among the plurality of imaging positions 8A.
  • a flight route 8 passing through the plurality of imaging positions 8A is set.
  • step ST228 the second zoom magnification determination unit 144 determines the zoom magnification of the imaging device 330 of the flying object 310 to be the second zoom magnification. After the process of step ST228 is executed, the flight imaging support process proceeds to step ST229.
  • step ST229 the second zoom magnification storage control section 146 causes the storage 52 to store the second zoom magnification determined by the second zoom magnification determination section 144.
  • FIG. After the process of step ST229 is executed, the flight imaging support process proceeds to step ST230.
  • step ST230 the second flight route setting section 148 sets a plurality of imaging positions 8A on the smooth surface 7 based on the second zoom magnification determined by the second zoom magnification determining section 144, thereby obtaining a plurality of A flight route 8 passing through the imaging position 8A is set. Even when the second zoom magnification is adjusted according to the distance between the inspection target surface 4G and the imaging position 8A in steps ST273 and ST274, which will be described later, the second flight route setting unit 148 sets a plurality of imaging positions 8A. By setting a plurality of imaging positions 8A at positions where the imaging ranges 331 of the imaging devices 330 partly overlap with each other at adjacent imaging positions 8A, a flight route 8 passing through the plurality of imaging positions 8A is set. After the process of step ST230 is executed, the flight imaging support process proceeds to step ST240.
  • step ST240 operation mode setting section 102 sets the operation mode of base station 10 to flight control mode. After the process of step ST240 is executed, the flight imaging support process proceeds to step ST241.
  • step ST241 the third reception determination unit 152 determines whether or not the instruction to start flight has been received by the reception device 14. In step ST241, if the instruction to start flight has not been received by the reception device 14, the determination is negative, and the determination in step ST241 is performed again. In step ST241, when the instruction to start flight is accepted by the accepting device 14, the determination is affirmative, and the flight imaging support process proceeds to step ST242.
  • step ST242 the second imaging control unit 154 causes the imaging device 30 of each base station 10 to image the imaging scene.
  • step ST242 the flight imaging support process proceeds to step ST242A.
  • step ST242A the first flying object determination unit 216 executes object recognition processing on the images obtained by the imaging device 30 of each base station 10, so that the first base station 10A and the first base station 10A It is determined which base station 10 of the two base stations 10B has an image in which the aircraft 310 is captured.
  • step ST242A if the first flying object determination unit 216 determines that the flying object 310 is shown as an image in the image obtained by the first base station 10A, the flight imaging support process proceeds to step ST243A. do.
  • step ST242A if the first flying object determination unit 216 determines that the flying object 310 is shown as an image in the image obtained by the second base station 10B, the flight imaging support process proceeds to step ST243B. do.
  • step ST243A the flying object position deriving unit 156 derives the position of the flying object 310 in the image obtained by being imaged by the imaging device 30 of the first base station 10A. After the process of step ST243A is executed, the flight imaging support process proceeds to step ST244A.
  • step ST244A the positional deviation determination unit 158 determines the position of the flying object 310 relative to the central portion of the angle of view of the imaging device 30 of the first base station 10A based on the position of the flying object 310 in the image derived in step ST243A. is out of position.
  • step ST244A if the position of flying object 310 is deviated from the central portion of the angle of view of first base station 10A, the determination is affirmative, and the flight imaging support process proceeds to step ST245A.
  • step ST244A if the position of flying object 310 is not shifted with respect to the central portion of the angle of view, the determination is negative, and the flight imaging support process proceeds to step ST246A.
  • step ST245A the second rotation control unit 160 adjusts the rotation angle of the rotation drive device 20 of the first base station 10A to an angle at which the aircraft 310 is positioned at the center of the angle of view of the imaging device 30.
  • step ST245A the flight imaging support process proceeds to step ST246A.
  • step ST243B the flying object position deriving unit 156 derives the position of the flying object 310 in the image obtained by being imaged by the imaging device 30 of the second base station 10B. After the process of step ST243 is executed, the flight imaging support process proceeds to step ST244B.
  • step ST244B based on the position of the flying object 310 in the image derived in step ST243B, the positional deviation determination unit 158 determines whether the flying object 310 is positioned relative to the central portion of the angle of view of the imaging device 30 of the second base station 10B. is out of position.
  • step ST244B if the position of flying object 310 is deviated from the central portion of the angle of view of second base station 10B, the determination is affirmative, and the flight imaging support process proceeds to step ST245B.
  • step ST244B if the position of flying object 310 is not shifted with respect to the central portion of the angle of view, the determination is negative, and the flight imaging support process proceeds to step ST246B.
  • step ST245B the second rotation control unit 160 adjusts the rotation angle of the rotation drive device 20 of the second base station 10B to an angle at which the aircraft 310 is positioned at the center of the angle of view of the imaging device 30.
  • step ST245B the flight imaging support process proceeds to step ST246B.
  • step ST246A the second ranging control section 162 causes the ranging device 40 of the first base station 10A to scan the ranging range 41 with laser light. In this case, since the flying object 310 is positioned within the ranging range 41 of the ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained.
  • step ST246A the flight imaging support process proceeds to step ST247A.
  • step ST247A the flying object coordinate derivation unit 164 calculates the rotation angle of the rotary drive device 20, the angle of the laser light emitted from the rangefinder 40 toward the flying object 310, and the flying object coordinates for the first base station 10A. Based on the distance between 310 and rangefinder 40, relative coordinates of flying object 310 with respect to the position of first base station 10A are derived. After the process of step ST247A is executed, the flight imaging support process proceeds to step ST248.
  • step ST246B the second ranging control section 162 causes the ranging device 40 of the second base station 10B to scan the ranging range 41 with laser light. In this case, since the flying object 310 is positioned within the ranging range 41 of the ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained.
  • step ST246B the flight imaging support process proceeds to step ST247B.
  • step ST247B the flying object coordinate deriving unit 164 calculates the rotation angle of the rotary drive device 20, the angle of the laser light emitted from the rangefinder 40 toward the flying object 310, and the flying object coordinates for the second base station 10B. Relative coordinates of the aircraft 310 with respect to the position of the first base station 10A are derived based on the distance between 310 and the rangefinder 40 and the calibration information.
  • step ST248 the flight imaging support process proceeds to step ST248.
  • step ST248 the imaging position arrival determination unit 166 determines whether the flying object 310 has reached the target imaging position 8A based on the coordinates of the flying object 310 derived in step ST247A or step ST247B and the coordinates of the target imaging position 8A. determine whether or not In step ST248, if the flying object 310 has not reached the target imaging position 8A, the determination is negative, and the flight imaging support process proceeds to step ST260. In step ST248, when the flying object 310 reaches the target imaging position 8A, the determination is affirmative, and the flight imaging support process proceeds to step ST249.
  • step ST249 the flight instruction generator 168 issues flight instructions to the flying object 310 based on the difference between the absolute coordinates of the flying object 310 derived in step ST247A or step ST247B and the absolute coordinates of the target imaging position 8A. Generate. After the process of step ST249 is executed, the flight imaging support process proceeds to step ST250.
  • step ST250 the flight instruction transmission control section 170 transmits flight instructions to the aircraft 310 via the communication device 12.
  • step ST250 the flight imaging support process proceeds to step ST241.
  • Steps ST241 to ST244B and steps ST246A to ST250 are repeatedly executed, and when the flying object 310 reaches the target imaging position 8A, the determination in step ST250 is affirmative, and the flight imaging support process The process proceeds to step ST260.
  • step ST260 the operation mode setting section 102 sets the operation mode of the base station 10 to the imaging control mode. After the process of step ST260 is executed, the flight imaging support process proceeds to step ST261.
  • step ST261 the hovering instruction transmission control section 172 transmits a hovering instruction to the aircraft 310 via the communication device 12. After the process of step ST261 is executed, the flight imaging support process proceeds to step ST262.
  • step ST262 the hovering report reception determination unit 174 determines whether or not the communication device 12 has received a hovering report. In step ST262, if no hovering report has been received by the communication device 12, the determination is negative, and the determination in step ST262 is performed again. In step ST262, when the hovering report is received by the communication device 12, the determination is affirmative, and the flight imaging support process proceeds to step ST263.
  • step ST263 the third imaging control unit 176 causes the imaging device 30 of each base station 10 to image the imaging scene.
  • step ST263 the flight imaging support process proceeds to step ST263A.
  • step ST263A the second flying object determination unit 218 performs object recognition processing on images obtained by imaging by the imaging device 30 of each base station 10, thereby performing object recognition processing on the first base station 10A and the first base station 10A. It is determined which base station 10 of the two base stations 10B has an image in which the aircraft 310 is captured.
  • step ST263A if the second flying object determination unit 218 determines that the flying object 310 is shown as an image in the image obtained by the first base station 10A, the flight imaging support process proceeds to step ST264A. do.
  • step ST263A if the first flying object determination unit 216 determines that the flying object 310 is shown as an image in the image obtained by the second base station 10B, the flight imaging support process proceeds to step ST264B. do.
  • step ST264A the aircraft attitude identifying section 178 performs object recognition processing on the image obtained by the first base station 10A, thereby determining the flight position based on the positions of the propellers 341 captured in the image. Identify the pose of the body 310 .
  • step ST264A the flight imaging support process proceeds to step ST265.
  • step ST264B the aircraft attitude identification section 178 performs object recognition processing on the image obtained by the second base station 10B, thereby determining the flight position based on the positions of the plurality of propellers 341 captured in the image. Identify the pose of the body 310 .
  • step ST264B the flight imaging support process proceeds to step ST265.
  • step ST265 the attitude correction instruction generating section 180 generates an attitude correction instruction for the flying object 310 based on the attitude of the flying object 310 identified at step ST264.
  • step ST265 the flight imaging support process proceeds to step ST266.
  • step ST266 the attitude correction instruction transmission control section 182 transmits an attitude correction instruction to the aircraft 310 via the communication device 12.
  • the flight imaging support process proceeds to step ST70 (see FIG. 38).
  • Steps ST70 to ST84 are the same as in the first embodiment. Note that in the second embodiment, if the determination in step ST83 (see FIG. 39) is negative, the flight imaging support process proceeds to step ST241.
  • the processor 51 rotates the distance measuring device 40 with respect to the rotary drive device 20 of the first base station 10A, and rotates the distance measuring device 40 of the first base station 10A. to measure the distances at a plurality of first distance measurement points on the wall surface 4 . Further, the processor 51 rotates the distance measuring device 40 with respect to the rotary drive device 20 of the second base station 10B, and causes the distance measuring device 40 of the second base station 10B to perform a plurality of second measurements of the wall surface 4. Have the distance measured for the distance point. Then, the processor 51 sets the flight route 8 based on the distance measured at each first ranging point and the distance measured at each second ranging point. Therefore, for example, a long flight route 8 can be set compared to setting the flight route 8 with one base station 10 .
  • processor 51 converts the distance measured by ranging device 40 of second base station 10B to the distance based on the position of ranging device 40 of first base station 10A based on predetermined calibration information. Convert. Therefore, for example, the flight route 8 can be set with respect to the ranging area of the ranging device 40 of the second base station 10B based on the position of the ranging device 40 of the first base station 10A.
  • the processor 51 calculates the position of the flying object 310 measured by the ranging device 40 of the second base station 10B based on the predetermined calibration information, and the position of the ranging device 40 of the first base station 10A as a reference. Convert to the position where Therefore, for example, when the flying object 310 flies in the ranging area of the ranging device 40 of the second base station 10B, the flying object 310 can be controlled based on the position of the first base station 10A.
  • the processor 51 selects the distance measuring device 40 of the first base station 10A and the distance measuring device 40 of the second base station 10B to measure the position of the flying object 310 according to the position of the flying object 310. to select. Therefore, for example, the flying object 310 that flies along the flight route 8 set from the ranging area of the ranging device 40 of the first base station 10A to the ranging area of the ranging device 40 of the first base station 10A is controlled. can do.
  • the imaging system S includes the first base station 10A and the second base station 10B as examples of the plurality of base stations, but may include three or more base stations.
  • the controller 60 has a distance derivation mode as an operation mode. While the flight route setting processing unit 104 is executing the flight route setting process, the operation mode setting unit 102 determines the distance between the point X located outside the range-finding area of each range-finding device 40 and each range-finding device 40 . A distance derivation mode is set as the operation mode of the controller 60 when deriving the distance between.
  • the operation mode setting unit 102 sets the distance between the point X located outside the range-finding area of each range-finding device 40 and each range-finding device 40 while the flight control processing unit 106 is executing the flight control processing.
  • a distance derivation mode is set as the operation mode of the base station 10.
  • the processor 51 operates as the distance derivation processing unit 220 .
  • the distance derivation processing section 220 has a rotation control section 222 and a distance derivation section 224 .
  • the third embodiment in contrast to the second embodiment, a point X positioned outside the range-finding area of each range-finding device 40 and each range-finding device 40 An example of deriving the distance between will be described.
  • the point X is the position of the inspection object 3 on the wall surface 4, which is the reference position for setting the flight route 8.
  • FIG. 61 the point X is the position of the inspection object 3 on the wall surface 4, which is the reference position for setting the flight route 8.
  • the ranging area is called a first ranging area
  • the ranging area of the ranging device 40 of the second base station 10B is called a second ranging area.
  • the first ranging area is an example of the "first ranging area” according to the technique of the present disclosure
  • the second ranging area is an example of the "second ranging area” according to the technique of the present disclosure.
  • the rotation control unit 222 controls each rotation drive device 20 to adjust the rotation angle of each rotation drive device 20 to an angle at which the point X is positioned at the center of the angle of view of each imaging device 30 .
  • the rotation control unit 222 controls each rotation drive device 20 based on the position designation instruction to perform each rotation drive.
  • the rotation angle of the device 20 is adjusted so that the point X on the wall surface 4 is positioned at the center of the angle of view of each imaging device 30 .
  • the rotation control unit 222 controls the rotation angle of the rotation drive device 20 of the first base station 10A.
  • the rotation angle of the rotation driving device 20 of the second base station 10B is adjusted so that the aircraft 310 is positioned at the center of the angle of view of the imaging device 30 of the second base station 10B. Adjust to the desired angle.
  • the rotation control unit 222 controls the rotation angle of the rotation drive device 20 of the second base station 10B.
  • the rotation angle of the rotation driving device 20 of the first base station 10A is adjusted so that the aircraft 310 is positioned at the center of the angle of view of the imaging device 30 of the first base station 10A. Adjust to the desired angle.
  • the rotation angle of the rotary drive device 20 of the first base station 10A is adjusted.
  • the rotation angle of the rotary drive device 20 is set to the angle in the direction in which the point X is positioned with respect to the distance measuring device 40 of the first base station 10A.
  • the second base station 10B by adjusting the rotation angle of the rotation drive device 20 of the second base station 10B to an angle at which the aircraft 310 is positioned in the center of the angle of view of the imaging device 30 of the second base station 10B, the second base station 10B The rotation angle of the rotary drive device 20 of the station 10B is set to the angle of the direction in which the point X is positioned with respect to the distance measuring device 40 of the second base station 10B.
  • the distance derivation unit 224 derives the distance between each distance measuring device 40 and the point X based on the calibration information and the rotation angle of each rotary drive device 20 . A procedure for deriving the distance between each distance measuring device 40 and the point X will be described below with reference to FIG.
  • the distance derivation unit 224 derives the angle ⁇ xc1 of the side X1 with the side C as a reference based on the calibration information and the rotation angle of the rotary drive device 20 of the first base station 10A.
  • a side X1 is a side connecting the point X and the point C1 of the first base station 10A.
  • the position of the first base station 10A is synonymous with the position of the ranging device 40 of the first base station 10A.
  • the distance derivation unit 224 derives the angle ⁇ xc2 of the side X2 with the side C as a reference based on the calibration information and the rotation angle of the rotary drive device 20 of the second base station 10B.
  • a side X2 is a side connecting the point X and the point C2 of the second base station 10B.
  • the position of the second base station 10B is synonymous with the position of the ranging device 40 of the second base station 10B.
  • the distance derivation unit 224 calculates the length Lx1 of the side X1 based on (9) below.
  • the distance derivation unit 224 calculates the length Lx2 of the side X2 based on (10) below. The distance between each distance measuring device 40 and the point X is derived by the above procedure.
  • step ST321 the rotation control unit 222 controls each rotation driving device 20 to set the rotation angle of each rotation driving device 20 to the center of the angle of view of each imaging device 30. Adjust the angle so that the point X is located in the part.
  • step ST322 the distance derivation unit 224 derives the distance between each distance measuring device 40 and the point X based on the calibration information and the rotation angle of each rotary drive device 20.
  • the processor 51 determines the first ranging area of the ranging device 40 of the first base station 10A and the second ranging area of the ranging device 40 of the second base station 10B.
  • the distance between the point X and the range finder 40 of the first base station 10A is calculated as It is derived based on the angle of the direction in which the point X is located and the distance between the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B.
  • the processor 51 calculates the distance between the point X and the ranging device 40 of the second base station 10B by the angle of the direction in which the point X is located with respect to the ranging device 40 of the second base station 10B, It is derived based on the distance between the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B. Therefore, the flight route 8 can be set based on the point X located outside the first range-finding area and the second range-finding area.
  • the processor 51 calculates the distance between the flying object 310 and the ranging device 40 of the first base station 10A as the first Based on the angle of the direction in which the flying object 310 is positioned with respect to the ranging device 40 of the first base station 10A and the distance between the ranging device 40 of the first base station 10A and the ranging device 40 of the second base station 10B to derive Similarly, the processor 51 calculates the distance between the flying object 310 and the ranging device 40 of the second base station 10B by the angle of the direction in which the flying object 310 is positioned with respect to the ranging device 40 of the second base station 10B.
  • the processor 51 executes the flight imaging support program 100 to perform the operation mode setting unit 102, the flight route setting processing unit 104, the flight control processing unit 106, and the imaging control processing unit 108, as well as the position correction processing unit. 230.
  • the base station 10 has, as operation modes, a flight route setting processing mode, a flight control processing mode, a position correction processing mode, and an imaging control processing mode.
  • the operation mode setting unit 102 sets a flight route setting processing mode, a flight control processing mode, a position correction processing mode, and an imaging control processing mode as the operation modes of the base station 10 .
  • the processor 51 operates as the position correction processing unit 230 .
  • the operation mode setting unit 102 switches from the flight control processing mode to the imaging control processing mode, but in the fourth embodiment, the operation mode setting unit 102 switches from the flight control processing mode to the imaging control processing mode. During the transition, set the position correction processing mode.
  • the position correction processing unit 230 includes an imaging instruction transmission control unit 232, an imaging report reception determination unit 234, an overlap amount derivation unit 236, a position correction amount derivation unit 238, a position correction instruction generation unit 240. , a position correction instruction transmission control unit 242, an imaging control unit 244, an aircraft position derivation unit 246, a position deviation determination unit 248, a rotation control unit 250, a ranging control unit 252, an aircraft coordinate derivation unit 254, and a position correction determination unit 256.
  • the imaging instruction transmission control unit 232 control to transmit imaging instructions to the flying object 310 via.
  • the imaging device 330 of the flying object 310 images the wall surface 4 according to the imaging instruction. As a result, a position correction image is obtained.
  • the flying object 310 transmits an imaging report to the base station 10 .
  • the imaging report includes the inspection image acquired in the previous imaging control process and the position correction image described above.
  • the inspection image acquired in the previous imaging control process will be referred to as the previous inspection image.
  • the imaging position 8A reached by the flying object 310 when the previous inspection image was acquired will be referred to as the previous imaging position 8A.
  • the previous inspection image is an image captured by the imaging device 330 under the control of the imaging instruction transmission control unit 198 (see FIG. 30) of the imaging control processing unit 108 in the imaging control processing mode. .
  • the imaging report reception determination unit 234 determines whether or not the communication device 12 has received the imaging report.
  • the overlap amount derivation unit 236 derives the overlap amount between the previous inspection image and the position correction image when the imaging report reception determination unit 234 determines that the communication device 12 has received the imaging report.
  • the position correction amount derivation unit 238 derives a position correction amount for correcting the position of the aircraft 310 with respect to the target imaging position 8A based on the overlap amount derived by the overlap amount derivation unit 236. In this case, the position correction amount derivation unit 238 calculates the position correction amount corresponding to the difference between the overlap amount derived by the overlap amount derivation unit 236 and the predetermined overlap amount. is derived based on the distance of The predetermined overlap amount is an amount that defines the amount of overlap between adjacent inspection images. The amount is set to allow recognition of adjacent inspection images.
  • the position correction instruction generation unit 240 generates a position correction instruction based on the position correction amount derived by the position correction amount derivation unit 238 .
  • the position correction instruction transmission control unit 242 performs control for transmitting a position correction instruction to the aircraft 310 via the communication device 12 .
  • the flying object 310 receives the position correction instruction as a flight instruction (see FIG. 22). Upon receiving the position correction instruction as the flight instruction, the flying object 310 changes its position by flying according to the position correction instruction.
  • the imaging control unit 244 controls the imaging device 30 to capture an imaging scene including the flying object 310 .
  • the flying object position deriving unit 246 performs object recognition processing on an image obtained by capturing an imaging scene including the flying object 310 with the imaging device 30, thereby determining the flying object 310 included in the image as an image. Derive the position in the image of
  • the positional deviation determination unit 248 determines whether the position of the flying object 310 is shifted from the central portion of the angle of view of the imaging device 30. determine whether or not
  • the rotation control unit 250 adjusts the horizontal rotation angle and/or the vertical rotation of the rotary drive device 20. Control is performed to adjust the angle so that the aircraft 310 is positioned at the center of the angle of view of the imaging device 30 .
  • the ranging control unit 252 controls the ranging device 40 to scan the ranging range 41 with laser light. In this case, since the flying object 310 is positioned within the ranging range 41 of the ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained.
  • the flying object coordinate deriving unit 254 calculates the absolute coordinates of the rotary drive device 20, the rotation angle of the rotary driving device 20, the angle of the laser beam emitted from the rangefinder 40 toward the flying object 310, and the flying object 310 and the measured value. Based on the distance to the range device 40, the absolute coordinates of the aircraft 310 are derived.
  • the position correction determining unit 256 determines whether or not the position of the flying object 310 has been corrected based on the absolute coordinates of the flying object 310 derived by the flying object coordinate deriving unit 254 .
  • the imaging instruction transmission control unit 232, the imaging report reception determination unit 234, the overlap amount deriving unit 236, the position correction amount A derivation unit 238, a position correction instruction generation unit 240, a position correction instruction transmission control unit 242, an imaging control unit 244, an aircraft position derivation unit 246, a position deviation determination unit 248, a rotation control unit 250, a ranging control unit 252, and a flight Processing by the body coordinate derivation unit 254 is executed. As a result, control is executed to fly the aircraft 310 to a position where the amount of overlap between the previous inspection image and the current inspection image is the predetermined overlap amount.
  • the imaging control processing mode is set as the operation mode of the base station 10, as in the first embodiment.
  • an inspection image is acquired in the current imaging control process.
  • the inspection image acquired in the current imaging control process will be referred to as the current inspection image.
  • the imaging position 8A reached by the aircraft 310 when the current inspection image is acquired will be referred to as the current imaging position 8A.
  • the operation mode of the base station 10 is an example of the "operation mode” according to the technology of the present disclosure.
  • the flight control processing mode is an example of the “first mode” according to the technology of the present disclosure
  • the position correction processing mode is an example of the "second mode” according to the technology of the present disclosure.
  • the imaging device 330 of the flying object 310 is an example of the “third imaging device” according to the technology of the present disclosure.
  • the phase correction image is an example of the "third image” according to the technology of the present disclosure.
  • the previous inspection image is an example of the "fourth image” according to the technology of the present disclosure. This inspection image is an example of the “fifth image” according to the technology of the present disclosure.
  • the previous imaging position 8A is an example of a "second imaging position” according to the technology of the present disclosure.
  • the current imaging position 8A is an example of a “third imaging position” according to the technology of the present disclosure.
  • the processing by the overlap amount derivation unit 236, that is, the processing for deriving the overlap amount between the previous inspection image and the position correction image may be executed by the processor 351 of the aircraft 310.
  • the overlap amount derived by the processor 351 of the aircraft 310 may then be transmitted to the processor 51 of the base station 10 .
  • FIG. 68 An example of the flow of position correction processing executed by the position correction processing section 230 according to the fourth embodiment will be described with reference to FIGS. 68 and 69.
  • FIG. 68 An example of the flow of position correction processing executed by the position correction processing section 230 according to the fourth embodiment will be described with reference to FIGS. 68 and 69.
  • step ST411 the imaging instruction transmission control unit 232 transmits an imaging instruction to the flying object 310 via the communication device 12.
  • step ST411 the position correction process proceeds to step ST412.
  • step ST412 the imaging report reception determination unit 234 determines whether or not the communication device 12 has received the imaging report. In step ST412, if the communication device 12 has not received the imaging report, the determination is negative, and the determination in step ST412 is performed again. In step ST412, when the communication device 12 receives the imaging report, the determination is affirmative, and the position correction process proceeds to step ST413.
  • step ST413 the overlap amount derivation unit 236 derives the overlap amount between the previous inspection image and the position correction image. After the process of step ST413 is executed, the position correction process proceeds to step ST414.
  • step ST414 the position correction amount derivation section 238 calculates the position correction amount corresponding to the difference between the overlap amount derived by the overlap amount derivation section 236 and the predetermined overlap amount. is derived based on the distance of After the process of step ST414 is executed, the position correction process proceeds to step ST415.
  • step ST415 the position correction instruction generation unit 240 generates a position correction instruction based on the position correction amount derived by the position correction amount derivation unit 238. After the process of step ST415 is executed, the position correction process proceeds to step ST416.
  • step ST416 the position correction instruction transmission control section 242 transmits a position correction instruction to the flying object 310 via the communication device 12. After the process of step ST416 is executed, the position correction process proceeds to step ST420.
  • step ST420 the imaging control unit 244 causes the imaging device 30 to capture an imaging scene including the flying object 310. After the process of step ST420 is executed, the position correction process proceeds to step ST421.
  • step ST421 the flying object position derivation unit 246 derives the position of the flying object 310 in the image obtained by being imaged by the imaging device 30. After the process of step ST421 is executed, the position correction process proceeds to step ST422.
  • step ST422 based on the position of the flying object 310 in the image derived in step ST421, the positional deviation determination unit 248 determines whether the position of the flying object 310 is shifted with respect to the central portion of the angle of view of the imaging device 30. determine whether or not In step ST422, if the position of the flying object 310 is deviated from the central portion of the angle of view, the determination is affirmative, and the position correction process proceeds to step ST423. In step ST422, if the position of flying object 310 is not shifted with respect to the central portion of the angle of view, the determination is negative, and the position correction process proceeds to step ST430.
  • step ST423 the rotation control unit 250 adjusts the rotation angle of the rotation drive device 20 to an angle at which the flying object 310 is positioned at the center of the angle of view of the imaging device 30.
  • step ST423 the position correction process proceeds to step ST430.
  • step ST430 the ranging control section 252 causes the ranging device 40 to scan the ranging range 41 with laser light. In this case, since the flying object 310 is positioned within the ranging range 41 of the ranging device 40, the distance between the flying object 310 and the ranging device 40 can be obtained. After the process of step ST430 is executed, the position correction process proceeds to step ST431.
  • step ST431 the flying object coordinate derivation unit 254 determines the absolute coordinates of the rotary drive device 20, the rotation angle of the rotary drive device 20, the angle of the laser light emitted from the rangefinder 40 toward the flying object 310, and the flight Based on the distance between the body 310 and the rangefinder 40, the absolute coordinates of the aircraft 310 are derived.
  • step ST431 the position correction process proceeds to step ST432.
  • the position correction determination unit 256 determines whether or not the position of the flying object 310 has been corrected based on the absolute coordinates of the flying object 310 derived at step ST431. In step ST432, if the position of flying object 310 has not been corrected, the determination is negative, and the position correction process proceeds to step ST420. In step ST432, if the position of the flying object 310 has been corrected, the determination is affirmative, and the position correction process ends.
  • the processor 51 uses the flight control processing mode in which the aircraft 310 flies based on the flight route 8 as the operation mode of the base station 10, and A position correction processing mode for correcting the position of the flying object 310 based on the position correction image obtained by imaging the wall surface 4 with the imaging device 330 when the flying object 310 reaches the current imaging position 8A. set. Then, the processor 51 causes the imaging device 330 to acquire the previous inspection image when the flying object 310 reaches the previous imaging position 8A, and then causes the flying object 310 to reach the current imaging position 8A.
  • the imaging device 330 In the position correction processing mode, when the imaging device 330 is caused to acquire the current inspection image in accordance with the fact that the previous inspection image The position of the aircraft 310 is corrected so that the amount of overlap between the current inspection image and the current inspection image becomes a predetermined overlap amount. Therefore, by correcting the position of the flying object 310, for example, when the flying object 310 reaches the current imaging position 8A, the imaging device 330 acquires the current inspection image. It is possible to improve the accuracy of the amount of overlap between the inspection image and the current inspection image.
  • imaging system S is used for inspection in the above embodiment, it may be used for purposes other than inspection, such as transportation, photography, surveying, pesticide spraying, maintenance, or security.
  • the base station 10 and the flying object 310 may perform the flight imaging support processing in a distributed manner.
  • the base station 10 and the external device may execute the flight imaging support processing in a distributed manner, the base station 10, the aircraft 310, and the external device may perform the flight imaging support processing in a distributed manner, and the aircraft 310 and the external device may perform the flight imaging support processing in a distributed manner.
  • the flight imaging assistance program 100 may be stored in a portable storage medium such as an SSD or USB memory.
  • a storage medium is a non-transitory computer-readable storage medium (ie, computer-readable storage medium).
  • a flight imaging support program 100 stored in a storage medium is installed in the computer 50 of the base station 10 .
  • the processor 51 of the base station 10 executes flight imaging support processing according to the flight imaging support program 100 .
  • the flight imaging program 400 may be stored in a portable storage medium such as an SSD or USB memory.
  • the storage medium is a non-transitory storage medium.
  • a flight imaging program 400 stored in a storage medium is installed in the computer 350 of the aircraft 310 .
  • the processor 351 of the flying object 310 executes flight imaging processing according to the flight imaging program 400 .
  • the flight imaging support program 100 is stored in a storage device such as another computer or server device connected to the base station 10 via the network, and the flight imaging support program 100 is stored in response to a request from the base station 10.
  • the support program 100 may be downloaded and installed on the computer 50 of the base station 10.
  • flight imaging support program 100 it is not necessary to store all of the flight imaging support program 100 in a storage device such as another computer or server device connected to the base station 10, or in the storage 52 of the base station 10. A part may be stored.
  • the flight imaging program 400 is stored in a storage device such as another computer or a server device connected to the aircraft 310 via the network, and the flight imaging program 400 is stored in response to a request from the aircraft 310 .
  • 400 may be downloaded and installed on computer 350 of air vehicle 310 .
  • flight imaging program 400 it is not necessary to store all of the flight imaging program 400 in a storage device such as another computer or server device connected to the flying object 310 or in the storage 352 of the flying object 310. may be stored.
  • the computer 50 is built in the base station 10, but the technology of the present disclosure is not limited to this, and the computer 50 may be provided outside the base station 10, for example.
  • the computer 350 is built in the flying object 310, but the technology of the present disclosure is not limited to this, and the computer 350 may be provided outside the flying object 310, for example.
  • the computer 50 is used in the base station 10, but the technology of the present disclosure is not limited to this, and a device including ASIC, FPGA, and/or PLD is applied instead of the computer 50. You may also, instead of the computer 50, a combination of hardware configuration and software configuration may be used.
  • the computer 350 is used in the aircraft 310, but the technology of the present disclosure is not limited to this, and instead of the computer 350, a device including ASIC, FPGA, and/or PLD is applied. You may also, instead of the computer 350, a combination of hardware configuration and software configuration may be used.
  • processors can be used as hardware resources for executing the various processes described in the above embodiments.
  • processors include CPUs, which are general-purpose processors that function as hardware resources that execute various processes by executing software, that is, programs.
  • the processor includes, for example, a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing specific processing such as FPGA, PLD, or ASIC.
  • a memory is built in or connected to each processor, and each processor executes processing by using the memory.
  • hardware resources for executing various processes may be configured with one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA).
  • the hardware resource that executes the processing may be one processor.
  • one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes.
  • this processor functions as a hardware resource that executes various processes.
  • SoC SoC, etc.
  • a and/or B is synonymous with “at least one of A and B.” That is, “A and/or B” means that only A, only B, or a combination of A and B may be used.
  • a and/or B means that only A, only B, or a combination of A and B may be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2022/019851 2021-06-29 2022-05-10 制御装置、基地局、制御方法、及びプログラム Ceased WO2023276454A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023531486A JPWO2023276454A1 (https=) 2021-06-29 2022-05-10
US18/534,713 US20240111311A1 (en) 2021-06-29 2023-12-10 Control apparatus, base station, control method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-108047 2021-06-29
JP2021108047 2021-06-29

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/534,713 Continuation US20240111311A1 (en) 2021-06-29 2023-12-10 Control apparatus, base station, control method, and program

Publications (1)

Publication Number Publication Date
WO2023276454A1 true WO2023276454A1 (ja) 2023-01-05

Family

ID=84692724

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/019851 Ceased WO2023276454A1 (ja) 2021-06-29 2022-05-10 制御装置、基地局、制御方法、及びプログラム

Country Status (3)

Country Link
US (1) US20240111311A1 (https=)
JP (1) JPWO2023276454A1 (https=)
WO (1) WO2023276454A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026078514A1 (en) * 2024-10-08 2026-04-16 Raedyne Systems Pty Ltd System and method for aerial image-based infrastructure inspection

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006027448A (ja) * 2004-07-16 2006-02-02 Chugoku Electric Power Co Inc:The 無人飛行体を利用した空撮方法及びその装置
JP2016111414A (ja) * 2014-12-03 2016-06-20 コニカミノルタ株式会社 飛行体の位置検出システム及び飛行体
US20170248969A1 (en) * 2016-02-29 2017-08-31 Thinkware Corporation Method and system for providing route of unmanned air vehicle
JP2019117127A (ja) * 2017-12-27 2019-07-18 株式会社トプコン 三次元情報処理部、三次元情報処理部を備える装置、無人航空機、報知装置、三次元情報処理部を用いた移動体制御方法および移動体制御処理用プログラム
JP2020201850A (ja) * 2019-06-12 2020-12-17 九州旅客鉄道株式会社 飛行体の制御方法

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6414760B1 (en) * 1998-10-29 2002-07-02 Hewlett-Packard Company Image scanner with optical waveguide and enhanced optical sampling rate
IL201682A0 (en) * 2009-10-22 2010-11-30 Bluebird Aero Systems Ltd Imaging system for uav
US12007763B2 (en) * 2014-06-19 2024-06-11 Skydio, Inc. Magic wand interface and other user interaction paradigms for a flying digital assistant
CN112026727A (zh) * 2015-05-12 2020-12-04 深圳市大疆创新科技有限公司 识别或检测障碍物的设备和方法
US11203425B2 (en) * 2016-06-30 2021-12-21 Skydio, Inc. Unmanned aerial vehicle inspection system
US12162160B2 (en) * 2016-12-23 2024-12-10 Gecko Robotics, Inc. System, apparatus and method for improved location identification with prism
US11064184B2 (en) * 2017-08-25 2021-07-13 Aurora Flight Sciences Corporation Aerial vehicle imaging and targeting system
JP7215576B2 (ja) * 2019-06-17 2023-01-31 日本電信電話株式会社 測位依頼方法、測位依頼装置および測位依頼プログラム
US20230106432A1 (en) * 2021-06-25 2023-04-06 Knightwerx Inc. Unmanned system maneuver controller systems and methods
US20240111305A1 (en) * 2022-04-05 2024-04-04 Colorblind Enterprises, Llc Unmanned aerial vehicle event response system and method
US20230315128A1 (en) * 2022-04-05 2023-10-05 Colorblind Enterprises, Llc Unmanned aerial vehicle event response system and method
US20230409054A1 (en) * 2022-04-05 2023-12-21 Colorblind Enterprises, Llc Unmanned aerial vehicle event response system and method
US20240142978A1 (en) * 2022-10-31 2024-05-02 Tempest Droneworx, Inc. Systems and Methods for Fusing Sensor Data from Drones in a Virtual Environment
KR20240166263A (ko) * 2023-05-17 2024-11-26 현대자동차주식회사 비행체를 이용한 차량의 점검 방법 및 이를 위한 비행체
US20250108895A1 (en) * 2023-10-01 2025-04-03 The Hong Kong University Of Science And Technology Modular underwater pipeline inspection device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006027448A (ja) * 2004-07-16 2006-02-02 Chugoku Electric Power Co Inc:The 無人飛行体を利用した空撮方法及びその装置
JP2016111414A (ja) * 2014-12-03 2016-06-20 コニカミノルタ株式会社 飛行体の位置検出システム及び飛行体
US20170248969A1 (en) * 2016-02-29 2017-08-31 Thinkware Corporation Method and system for providing route of unmanned air vehicle
JP2019117127A (ja) * 2017-12-27 2019-07-18 株式会社トプコン 三次元情報処理部、三次元情報処理部を備える装置、無人航空機、報知装置、三次元情報処理部を用いた移動体制御方法および移動体制御処理用プログラム
JP2020201850A (ja) * 2019-06-12 2020-12-17 九州旅客鉄道株式会社 飛行体の制御方法

Also Published As

Publication number Publication date
JPWO2023276454A1 (https=) 2023-01-05
US20240111311A1 (en) 2024-04-04

Similar Documents

Publication Publication Date Title
KR102669474B1 (ko) 항공기를 위한 레이저 스페클 시스템
JP3600230B2 (ja) 建築および土木構造物計測・解析システム
US12235191B2 (en) Flight imaging system and method
US9989357B2 (en) Aerial device that cooperates with an external projector to measure three-dimensional coordinates
CN109073498B (zh) 用于对象的损坏分析的系统
ES2976466T3 (es) Sistema de detección de defectos usando un UAV equipado con cámara para fachadas de edificios en geometría de inmuebles complejos con una trayectoria de vuelo óptima y desprovista automáticamente de conflictos con obstáculos
JP6560366B2 (ja) 構造物の部材特定装置及び方法
US7697126B2 (en) Three dimensional spatial imaging system and method
BR102018012662B1 (pt) Método de operação de um veículo aéreo não tripulado e veículo aéreo não tripulado
JP5911934B2 (ja) 輪郭線計測装置およびロボットシステム
EP2863174B1 (en) Measuring method and measuring instrument
JP2016111414A (ja) 飛行体の位置検出システム及び飛行体
US20220307834A1 (en) Surveying System
CN113984019A (zh) 测量系统、测量方法以及测量用程序
US12524870B2 (en) Bar arrangement inspection system and bar arrangement inspection method
JP4565348B2 (ja) 形状測定装置及びその方法
WO2023276454A1 (ja) 制御装置、基地局、制御方法、及びプログラム
US20240176025A1 (en) Generating a parallax free two and a half (2.5) dimensional point cloud using a high resolution image
JP2006209318A (ja) 人数検出装置及び方法
JP2023038693A (ja) 測量データ処理装置、測量データ処理方法および測量データ処理用プログラム
CN118598007A (zh) 一种基于实景建模的塔式起重机变幅位置修正系统
WO2022259799A1 (ja) 制御装置、飛行体システム、制御方法、及びプログラム
WO2024118396A1 (en) Generating a parallax free two and a half (2.5) dimensional point cloud using a high resolution image
CN112304250B (zh) 一种移动物体之间的三维匹配设备及方法
WO2022078437A1 (zh) 一种移动物体之间的三维处理设备及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22830375

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023531486

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22830375

Country of ref document: EP

Kind code of ref document: A1