Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
  • H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
  • H04N23/71—Circuitry for evaluating the brightness variation
• • G—PHYSICS
  • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
  • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
  • G03B15/00—Special procedures for taking photographs; Apparatus therefor
  • G03B15/02—Illuminating scene
  • G03B15/03—Combinations of cameras with lighting apparatus; Flash units
• • G—PHYSICS
  • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
  • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
  • G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
  • H04N23/72—Combination of two or more compensation controls
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/70—Circuitry for compensating brightness variation in the scene
  • H04N23/73—Circuitry for compensating brightness variation in the scene by influencing the exposure time
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B64—AIRCRAFT; AVIATION; COSMONAUTICS
  • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
  • B64U10/00—Type of UAV
  • B64U10/10—Rotorcrafts
  • B64U10/13—Flying platforms
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B64—AIRCRAFT; AVIATION; COSMONAUTICS
  • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
  • B64U2101/00—UAVs specially adapted for particular uses or applications
  • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography

Definitions

• the technology of the present disclosure relates to a control device, a control method, and a program.
• the image processing device includes an image input section, a damage detection section, an image judgment section, a display control section, and a detection result correction section.
• the image input unit inputs a plurality of images obtained by dividing a subject into images.
• the damage detection unit detects damage to the subject from individual images that constitute a plurality of images.
• the image determination unit determines whether or not the individual image is to be a confirmation target image that allows the user to confirm the detection result for the individual image.
• the display control unit causes the display device to display a partial image obtained by cutting out the confirmation target image or a partial area of the confirmation target image to fit the display area of the display device in association with the detection result in the confirmation target image or partial image.
• the detection result modification unit modifies the detection result based on a user's instruction input.
• Japanese Unexamined Patent Publication No. 2020-155902 discloses an imaging device that images a subject while being installed on a moving body.
• the imaging device includes a plurality of illumination imaging units each capturing images of different ranges on the subject in a direction intersecting the moving direction of the moving body.
• Each of the plurality of illumination imaging units includes an illumination unit that illuminates at least a part of the illumination range on the subject, and an imaging unit that images an area included in the illumination range.
• the imaging device images a non-overlapping region on a subject where respective illumination ranges of the plurality of illumination imaging units do not overlap.
• JP 2022-007039A discloses an imaging device including two light sources, an image sensor, a control section, a specifying section, and a processing section.
• the image sensor images a subject using light emitted from two light sources and reflected by the subject.
• the control unit controls the light emission timing of each of the two light sources and the exposure timing of the image sensor, thereby obtaining a first image of the subject captured by the image sensor using only the first light source of the two light sources;
• a second image of the subject is captured by the image sensor using only the second light source of the two light sources.
• the identification unit identifies an area where blown-out highlights have occurred in the first image as an area where blown-out highlights have occurred.
• the processing unit generates a composite image by combining an image of the first image other than the blown-out highlight area and an image of the area corresponding to the blown-out highlight area of the second image, and processes the generated composite image. Output.
• One embodiment of the technology of the present disclosure includes, for example, a control device and a control method that can obtain an image that contributes to ensuring appropriate brightness of the entire composite image as an image used to generate the composite image, and provide programs.
• a first aspect of the technology of the present disclosure is a control device that is applied to a moving body equipped with an imaging device and a light source, in which the imaging device selects one of the subjects according to the moving position of the moving body.
• a first imaging target area and a second imaging target area are imaged, and the first area, which is a part of the first imaging target area, overlaps the second area, which is a part of the second imaging target area.
• the control device includes a processor, the processor controls the light source to irradiate light to the first imaging target area, and the intensity of the light irradiated to the first area is determined by the intensity of the light irradiated to the first area. This is a control device that has a higher intensity than the light irradiated to the surrounding area.
• a second aspect according to the technology of the present disclosure is that in the control device according to the first aspect, the processor is configured to combine a first captured image obtained by capturing the first imaging target region and a second imaging target region.
• a composite image is generated by combining the second captured image obtained by the imaging, and when the composite image is generated, an overlap image corresponding to the first region and the second region of the composite image.
• the control device uses a pixel value of an image area corresponding to a second area of the second captured image as a pixel value of the area.
• a third aspect according to the technology of the present disclosure is a control device according to the first aspect or the second aspect, in which the control is performed on a first captured image obtained by capturing the first imaging target area.
• This is a control device including control to fit an image of a light source in an image area corresponding to a first area of the image area.
• a fourth aspect of the technology of the present disclosure is a control device according to any one of the first to third aspects, wherein the imaging device includes an image sensor having a light receiving surface, and the processor includes an image sensor having a light receiving surface.
• the control device adjusts the exposure amount of the light receiving surface based on the brightness of a third area other than the first area of the first imaging area when one imaging area is imaged by an imaging device.
• a fifth aspect according to the technology of the present disclosure is a control device according to the fourth aspect, in which the exposure amount is adjusted by changing the irradiation position and/or intensity of light.
• a sixth aspect according to the technology of the present disclosure is a control device according to the fourth aspect, in which the exposure amount is adjusted by changing the exposure to the light receiving surface.
• a seventh aspect of the technology of the present disclosure is a control device according to any one of the fourth to sixth aspects, in which the exposure amount is obtained by imaging the first imaging target area.
• the control device is configured to set an exposure amount such that a pixel value of an image area corresponding to the first area of the first captured image is equal to or greater than a first predetermined pixel value.
• An eighth aspect according to the technology of the present disclosure is that in the control device according to the seventh aspect, the first predetermined pixel value is set to a pixel value at which pixel values of at least a part of the image area are saturated. It is a control device.
• a ninth aspect according to the technology of the present disclosure is that in the control device according to the seventh aspect or the eighth aspect, the first imaging target area and the second imaging target area are arranged such that the first imaging target area and the second imaging target area are
• the processor obtains the length of the first region along the moving direction of the moving object based on the pixel values of the image region, and determines the length of the first region from the first region based on the length.
• This is a control device that moves the movable body to a position where it overlaps with the second area.
• a tenth aspect of the technology of the present disclosure is the control device according to any one of the seventh to ninth aspects, wherein the exposure amount corresponds to a third region of the first captured image.
• the control device is configured to set an exposure amount such that a pixel value of an image area is greater than or equal to a second predetermined pixel value and less than a first predetermined pixel value.
• An eleventh aspect according to the technology of the present disclosure is the control device according to the tenth aspect, in which the amount of overlap between the first region and the second region is a predetermined amount of overlap. .
• a twelfth aspect of the technology of the present disclosure is the control device according to any one of the first to eleventh aspects, wherein the light source is arranged in front of the imaging device in the moving direction of the moving body.
• the first imaging target area and the second imaging target area are the control device which are the areas imaged in the order of the first imaging target area and the second imaging target area.
• a thirteenth aspect according to the technology of the present disclosure is a control device according to any one of the first to eleventh aspects, wherein the light source is located on the rear side of the imaging device in the moving direction of the moving body.
• the first imaging target area and the second imaging target area are regions that are imaged in the order of the second imaging target area and the first imaging target area.
• a fourteenth aspect according to the technology of the present disclosure is that in the control device according to any one of the first to thirteenth aspects, the processor is configured to control when the second imaging target area is imaged by the imaging device. , the light source is controlled to irradiate light to the second imaging target area, and the intensity of the light irradiated to the fourth area of the second imaging target area on the opposite side to the second area is determined by the second imaging target area.
• This control device has a higher intensity than the light irradiated around the four areas.
• a fifteenth aspect according to the technology of the present disclosure is a control device according to any one of the first to fourteenth aspects, in which the light is irradiated along the normal direction of the first area. It is a device.
• a sixteenth aspect according to the technology of the present disclosure is a control device according to any one of the first to fifteenth aspects, in which the control is performed to a position where the first imaging target area is imaged by the imaging device.
• This is a control device that controls a light source to emit light when a moving body moves.
• a seventeenth aspect according to the technology of the present disclosure is a control device according to any one of the first to fifteenth aspects, wherein the control is directed to a position where the first imaging target area is imaged by the imaging device.
• This is a control device that controls a light source to emit light when a moving object moves.
• An eighteenth aspect of the technology of the present disclosure is a control method applied to a moving body equipped with an imaging device and a light source, wherein the imaging device selects one of the subjects according to the moving position of the moving body.
• a first imaging target area and a second imaging target area are imaged, and the first area, which is a part of the first imaging target area, overlaps the second area, which is a part of the second imaging target area.
• the control method includes controlling a light source to irradiate light to a first imaging target area, and the intensity of the light irradiated to the first area is set to be different from the surroundings of the first area. This is a control method that is higher than the intensity of the irradiated light.
• a nineteenth aspect of the technology of the present disclosure is a program for causing a computer to execute a process applied to a moving body equipped with an imaging device and a light source, the imaging device A first imaging target area and a second imaging target area of the subject are imaged according to the subject, and the first area, which is a part of the first imaging target area, is a part of the second imaging target area.
• the processing includes controlling the light source to irradiate light to the first imaging target area, and the intensity of the light irradiated to the first area is equal to or greater than the first imaging target area.
• This program is higher than the intensity of the light irradiated around the area.
• FIG. 3 is a front view showing an example of a mode in which a plurality of imaging target areas are sequentially imaged by the flight imaging device according to the first embodiment.
• FIG. 1 is a block diagram illustrating an example of the hardware configuration of the flight imaging device according to the first embodiment.
• FIG. 1 is a block diagram illustrating an example of the hardware configuration of an imaging device according to a first embodiment.
• FIG. 3 is a two-sided view showing an example of a mode in which an N-th imaging target region and an N+1-th imaging target region are imaged according to the flight position of the flight imaging device according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of a functional configuration for realizing flight imaging processing according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of operations of a first arrival determination section, a first imaging control section, a first brightness acquisition section, and a first exposure amount derivation section according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of operations of a first exposure amount deriving section, a second imaging control section, and a width acquisition section according to the first embodiment.
• FIG. 3 is a block diagram illustrating an example of operations of a width acquisition section, an imaging position correction section, and a second arrival determination section according to the first embodiment.
• FIG. 3 is a block diagram illustrating an example of operations of a second arrival determination section, a third imaging control section, a second brightness acquisition section, and a second exposure amount derivation section according to the first embodiment.
• FIG. 3 is a block diagram illustrating an example of the operation of a second exposure amount deriving section and a fourth imaging control section according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of the operation of an end determination section and a lights-out control section according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of a functional configuration for realizing composite image generation processing according to the first embodiment.
• FIG. 2 is a block diagram illustrating an example of the operation of an image acquisition section and an image composition section according to the first embodiment.
• FIG. 7 is a block diagram illustrating an example of operations of a first exposure amount deriving unit, a light source control unit, and a second imaging control unit according to the second embodiment.
• FIG. 7 is a two-sided view showing an example of a mode in which an Nth imaging target region and an N+1st imaging target region are imaged according to the flight position of the flight imaging device according to the third embodiment.
• It is a block diagram showing an example of operation of the 2nd exposure amount derivation part, the light source control part, and the 4th imaging control part concerning a 3rd embodiment.
• FIG. 7 is a block diagram illustrating an example of the operation of an image acquisition unit and an image synthesis unit according to a third embodiment.
• FIG. 7 is a front view showing an example of a mode in which a plurality of imaging target areas are sequentially imaged by the flight imaging device according to the fourth embodiment.
• FIG. 12 is a two-sided view showing an example of a mode in which the flight imaging device according to the fourth embodiment images an N-th imaging target region and an N+1-th imaging target region when flying to the first side.
• FIG. 12 is a two-sided view showing an example of a mode in which the flight imaging device according to the fourth embodiment images an N-th imaging target region and an N+1-th imaging target region when flying to the second side.
• FIG. 12 is a two-sided view showing an example of a mode in which the flight imaging device according to the fourth embodiment images an N-th imaging target region and an N+1-th imaging target region when flying to the second side.
• FIG. 12 is a front view showing an example of a mode in which the flight imaging device according to the fifth embodiment sequentially images a plurality of imaging target areas while flying an Nth flight route.
• FIG. 13 is a front view showing an example of a mode in which the flight imaging device according to the fifth embodiment sequentially images a plurality of imaging target areas while flying an N+1-th flight route.
• FIG. 12 is a block diagram showing an example of the operation of an image acquisition section and an image composition section according to a fifth embodiment.
• FIG. 12 is a front view showing an example of a mode in which the flight imaging device according to the sixth embodiment sequentially images a plurality of imaging target areas while flying an Nth flight route.
• FIG. 12 is a front view showing an example of a mode in which the flight imaging device according to the sixth embodiment sequentially images a plurality of imaging target areas while flying an N+1-th flight route.
• FIG. 12 is a block diagram showing an example of the operation of an image acquisition section and an image composition section according to a sixth embodiment.
• I/F is an abbreviation for "Interface”.
• RAM is an abbreviation for "Random Access Memory.”
• CPU is an abbreviation for "Central Processing Unit.”
• GPU is an abbreviation for “Graphics Processing Unit.”
• HDD is an abbreviation for “Hard Disk Drive.”
• SSD is an abbreviation for “Solid State Drive.”
• DRAM is an abbreviation for "Dynamic Random Access Memory.”
• SRAM is an abbreviation for "Static Random Access Memory.”
• GNSS is an abbreviation for “Global Navigation Satellite System.”
• GPS is an abbreviation for “Global Positioning System.”
• LiDAR is an abbreviation for "light detection and ranging.”
• NVM is an abbreviation for "Non-Volatile Memory.”
• ASIC is an abbreviation for “Application Specific Integrated Circuit.”
• FPGA is an abbreviation for “Field-Programmable Gate Array.”
• CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor.”
• CCD is an abbreviation for “Charge Coupled Device”.
• RGB is an abbreviation for “Red Green Blue.”
• CIE is an abbreviation for "Commission Internationale de l'Eclairage”.
• TPU is an abbreviation for "Tensor Processing Unit”.
• USB is an abbreviation for "Universal Serial Bus.”
• SoC is an abbreviation for "System-on-a-Chip.”
• IC is an abbreviation for "Integrated Circuit.”
• the term “vertical direction” refers to not only a complete vertical direction but also a vertical direction that is generally accepted in the technical field to which the technology of the present disclosure belongs, and which is within the spirit of the technology of the present disclosure. Refers to the vertical direction, including a certain amount of error.
• the term “horizontal direction” refers to not only a completely horizontal direction but also a horizontal direction that is generally accepted in the technical field to which the technology of the present disclosure belongs, and which is within the spirit of the technology of the present disclosure. Refers to the horizontal direction, including a degree of error that does not deviate.
• a quadrilateral refers to a perfect quadrilateral as well as a quadrilateral that is generally accepted in the technical field to which the technology of the present disclosure belongs, to the extent that it does not go against the spirit of the technology of the present disclosure. Refers to a rectangle that includes the error of .
• vertical means not only completely vertical but also perpendicular to a degree that is generally accepted in the technical field to which the technology of the present disclosure belongs and does not go against the spirit of the technology of the present disclosure. It refers to vertical in the sense of including the error of.
• constant means not only a complete constant but also a constant that is generally accepted in the technical field to which the technology of the present disclosure belongs, and that does not go against the spirit of the technology of the present disclosure. It refers to a constant in the sense of including the error of.
• overlap refers to not only a complete overlap, but also an overlap that is generally accepted in the technical field to which the technology of the present disclosure belongs, and that is consistent with the spirit of the technology of the present disclosure. It refers to overlap in the sense that it includes an error that does not conflict.
• Center means not only a perfect center but also a center that is generally accepted in the technical field to which the technology of the present disclosure belongs, and includes a degree of error that does not go against the spirit of the technology of the present disclosure. point to the center of In the description of this specification, the term “maximum value” refers to an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the perfect maximum value, and is contrary to the spirit of the technology of the present disclosure. Refers to the highest value, including the degree of error that does not occur.
• the term “minimum value” refers to an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the absolute minimum value, and is contrary to the spirit of the technology of the present disclosure. This refers to the lowest value, including the degree of error that does not occur.
• the term “center value” refers to an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the perfect center value, and is contrary to the spirit of the technology of the present disclosure. It refers to the central value in the sense that it includes a certain degree of error.
• the term “average value” refers to an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to a perfect average value, and is contrary to the spirit of the technology of the present disclosure. This refers to the average value, which includes a certain level of error.
• the flight imaging device 10 has a flight function and an imaging function, and images the wall surface 2A of the object 2 while flying.
• the concept of "flight” includes not only the meaning that the flying imaging device 10 moves in the air, but also the meaning that the flying imaging device 10 stands still in the air.
• the flying imaging device is an example of a “mobile imaging device” according to the technology of the present disclosure.
• the object 2 having the wall surface 2A is a pier provided on a bridge.
• the piers are made of reinforced concrete, for example.
• a bridge pier is mentioned here as an example of the target object 2
• the target object 2 may be an object other than a bridge pier (for example, a tunnel or a dam).
• the flight imaging device 10 includes a flying object 20 and an imaging device 60.
• the flying object 20 is, for example, an unmanned aircraft such as a drone.
• the flight function of the flight imaging device 10 is realized by the flying object 20.
• the flying object 20 has a plurality of propellers 42, and flies as the plurality of propellers 42 rotate. Flying the flying object 20 is synonymous with flying the flying imaging device 10.
• the flying object 20 is an example of a "mobile object" according to the technology of the present disclosure.
• the imaging device 60 is, for example, a digital camera or a video camera.
• the imaging function of the flight imaging device 10 is realized by the imaging device 60.
• the imaging device 60 is mounted on the flying object 20. As an example, the imaging device 60 is provided at the bottom of the flying object 20.
• the imaging device 60 is an example of an "imaging device" according to the technology of the present disclosure.
• a plurality of flight routes 4 are set for the target object 2.
• the plurality of flight routes 4 are set on a virtual plane facing the wall surface 2A.
• the plurality of flight routes 4 are lined up in the vertical direction, and each flight route 4 extends in the horizontal direction.
• the flight imaging device 10 stores flight route information 122 (see FIG. 6) indicating each flight route 4 as described later, and receives flight instruction signals from a transmitter (not shown) or a base station (not shown), etc. It is possible to autonomously fly each flight route 4 without depending on the aircraft.
• the flight imaging device 10 sequentially flies a plurality of flight routes 4.
• the flight imaging device 10 moves in the horizontal direction by flying each flight route 4 .
• the plurality of flight routes 4 are flight routes that cause the flight imaging device 10 to fly in the same direction.
• each flight route 4 is a flight route in which the flight imaging device 10 is flown from left to right toward the wall surface 2A.
• a plurality of imaging positions 5 are set on each flight route 4. At each imaging position 5, the flight imaging device 10 images the wall surface 2A.
• the flight imaging device 10 flies each flight route 4 independently, the flight imaging device 10 flies each flight route 4 based on a flight instruction signal from a transmitter or a base station. You may fly flight route 4.
• the flight imaging device 10 sequentially images a plurality of imaging target regions 3 of the wall surface 2A by imaging the wall surface 2A at each imaging position 5. Each imaging target area 3 corresponds to each imaging position 5.
• the imaging target area 3 is an area determined by the angle of view of the flight imaging device 10. In the example shown in FIG. 1, a rectangular area is shown as an example of the imaging target area 3.
• a plurality of imaging target regions 3 arranged in the horizontal and vertical directions are imaged.
• the plurality of imaging target regions 3 are imaged in such a manner that adjacent imaging target regions 3 in the horizontal or vertical direction partially overlap each other.
• FIG. 1 shows a mode in which each imaging target area 3 is imaged by the imaging device 60 in a state where the optical axis OA (see FIG. 3) of the imaging device 60 is perpendicular to the wall surface 2A.
• the following description will be given on the premise that each imaging target area 3 is imaged by the imaging device 60 in a state where the optical axis OA of the imaging device 60 is perpendicular to the wall surface 2A. Further, for convenience, the following description will be made on the assumption that the distance between the wall surface 2A and the imaging device 60 is constant.
• a plurality of images for synthesis 142 are obtained by sequentially capturing images of the plurality of imaging target regions 3 by the imaging device 60.
• a composite image 140 is generated by combining a plurality of images for composition 142.
• the plurality of images for synthesis 142 are synthesized so that parts of the images for synthesis 142 adjacent to each other in the horizontal or vertical direction overlap.
• overlap the fact that adjacent imaging target regions 3 partially overlap with each other, and that adjacent compositing images 142 partially overlap with each other may be respectively referred to as "overlap.”
• An example of the composite image 140 is a two-dimensional panoramic image.
• the two-dimensional panoramic image is just an example, and a three-dimensional image (for example, a three-dimensional panoramic image) is generated as the composite image 140 in the same manner as a two-dimensional panoramic image is generated as the composite image 140. You may also do so.
• the composite image 140 is used, for example, to inspect and/or survey the wall surface 2A of the object 2.
• the flying object 20 includes a flight device 22, an input/output I/F 24, a computer 26, a positioning unit 28, and an acceleration sensor 30.
• the computer 26 is an example of a “control device” and a “computer” according to the technology of the present disclosure.
• the computer 26 includes a processor 34, a storage 36, and a RAM 38.
• the processor 34, storage 36, and RAM 38 are interconnected via a bus 40, and the bus 40 is connected to the input/output I/F 24.
• the processor 34 includes, for example, a CPU, and controls the entire flight imaging device 10. Although an example in which the processor 34 includes a CPU is given here, this is just an example.
• processor 34 may include a CPU and a GPU. In this case, for example, the GPU operates under the control of the CPU and is responsible for executing image processing.
• the processor 34 is an example of a "processor" according to the technology of the present disclosure.
• the storage 36 is a nonvolatile storage device that stores various programs, various parameters, and the like. Examples of the storage 36 include an HDD and an SSD. Note that the HDD and SSD are just examples, and flash memory, magnetoresistive memory, and/or ferroelectric memory may be used instead of or in conjunction with the HDD and/or SSD. good.
• the RAM 38 is a memory in which information is temporarily stored, and is used by the processor 34 as a work memory. Examples of the RAM 38 include DRAM and/or SRAM.
• the flight device 22 has multiple propellers 42, multiple motors 44, and a motor driver 46.
• the motor driver 46 is connected to the processor 34 via the input/output I/F 24 and the bus 40. Motor driver 46 individually controls multiple motors 44 according to instructions from processor 34 .
• the number of multiple motors 44 is the same as the number of multiple propellers 42.
• a propeller 42 is fixed to the rotating shaft of each motor 44. Each motor 44 rotates a propeller 42 .
• the aircraft 20 flies as the plurality of propellers 42 rotate. Note that the number of the plurality of propellers 42 (in other words, the number of the plurality of motors 44) provided in the aircraft 20 is four, as an example, but this is just an example, and the number of the plurality of propellers 42 is , for example, there may be three or five or more.
• the positioning unit 28 is a device that detects the position of the flying object 20.
• the position of the aircraft 20 is detected using, for example, GNSS (eg, GPS).
• the positioning unit 28 includes a GNSS receiver (not shown).
• a GNSS receiver receives, for example, radio waves transmitted from multiple satellites.
• the positioning unit 28 detects the position of the flying object 20 based on the radio waves received by the GNSS receiver, and outputs positioning data (for example, data indicating latitude, longitude, and altitude) according to the detected position.
• the acceleration sensor 30 detects the acceleration of the aircraft 20 in the pitch, yaw, and roll axis directions.
• the acceleration sensor 30 outputs acceleration data according to the acceleration of the flying object 20 in each axial direction.
• Processor 34 obtains the position of aircraft 20 based on positioning data and/or acceleration data.
• the acceleration sensor 30 may be omitted.
• positioning unit 28 may be omitted.
• the processor 34 acquires the position of the aircraft 20 based on the positioning data
• the position in the absolute coordinate system is derived based on the positioning data.
• the processor 34 acquires the imaging position 5 based on the acceleration data
• the amount of change in position with respect to the reference position defined in the relative coordinate system is derived based on the acceleration data.
• the flying object 20 also includes other devices for detecting the position of the flying object 20 instead of or in addition to the positioning unit 28 and/or the acceleration sensor 30. You can leave it there.
• Other devices include, for example, a LiDAR scanner, a stereo camera, a magnetic compass, a barometric altimeter, or an ultrasonic sensor.
• the flight imaging device 10 has a lighting function.
• the illumination function of the flight imaging device 10 is a function of irradiating light onto the wall surface 2A (see FIG. 1).
• the flight imaging device 10 includes a light source 48 , and the illumination function of the flight imaging device 10 is realized by the light source 48 .
• the light source 48 is, for example, an LED light source, a laser light source, a strobe light source, or an incandescent light bulb.
• the light source 48 is an example of a "light source" according to the technology of the present disclosure.
• the imaging device 60 includes an imaging lens 62, an aperture 64, an aperture actuator 66, a shutter 68, a shutter actuator 70, a controller 72, an image sensor 74, and an image sensor driver 76.
• the controller 72 and the image sensor driver 76 are connected to the processor 34 via the input/output I/F 24 and the bus 40.
• the imaging lens 62 includes, for example, an objective lens (not shown), a focus lens (not shown), and the like.
• the imaging device 60 also includes a zoom lens (not shown). The imaging lens 62 is arranged on the object side with respect to the aperture 64, and the zoom lens is arranged between the aperture 64 and the shutter 68.
• the aperture actuator 66 has a power transmission mechanism (not shown) and an aperture motor (not shown).
• the diaphragm 64 has an aperture 64A, and the size of the aperture 64A is variable.
• the opening 64A is formed by a plurality of blades (not shown).
• the plurality of blades are connected to a power transmission mechanism.
• An aperture motor is connected to the power transmission mechanism, and the power transmission mechanism transmits the power of the aperture motor to the plurality of blades.
• the plurality of blades operate in response to power transmitted from the power transmission mechanism, thereby changing the size of the opening 64A.
• the aperture 64 adjusts exposure by changing the size of the aperture 64A.
• Aperture actuator 66 is connected to controller 72 .
• the controller 72 is, for example, a device that includes a computer including a CPU, NVM, RAM, and the like. Note that although a computer is illustrated here, this is just an example, and a device including an ASIC, an FPGA, and/or a PLD may be applied. Further, as the controller 72, for example, a device realized by a combination of a hardware configuration and a software configuration may be used. Controller 72 controls aperture actuator 66 according to instructions from processor 34 .
• the image sensor 74 includes a photoelectric conversion element 78 and a signal processing circuit 80.
• the image sensor 74 is, for example, a CMOS image sensor. Although a CMOS image sensor is exemplified here as the image sensor 74, the technology of the present disclosure is not limited to this. For example, even if the image sensor 74 is another type of image sensor such as a CCD image sensor, the present disclosure The technology is established.
• the photoelectric conversion element 78 is connected to the image sensor driver 76.
• the image sensor driver 76 controls the photoelectric conversion element 78 according to instructions from the processor 34.
• the photoelectric conversion element 78 has a light receiving surface 78A on which a plurality of pixels (not shown) are provided.
• the photoelectric conversion element 78 outputs the electrical signals output from the plurality of pixels to the signal processing circuit 80 as image data.
• the signal processing circuit 80 digitizes analog imaging data input from the photoelectric conversion element 78.
• the signal processing circuit 80 is connected to the input/output I/F 24.
• the digitized imaging data is subjected to various processing by the processor 34.
• the shutter 68 is, for example, a focal plane shutter, and is arranged between the aperture 64 and the light receiving surface 78A.
• the shutter 68 includes a front curtain 68A and a rear curtain 68B.
• each of the front curtain 68A and the rear curtain 68B includes a plurality of blades (not shown).
• the front curtain 68A is arranged closer to the subject than the rear curtain 68B.
• the shutter actuator 70 includes a link mechanism (not shown), a leading curtain solenoid (not shown), and a trailing curtain solenoid (not shown).
• the front curtain solenoid is a drive source for the front curtain 68A, and is mechanically connected to the front curtain 68A via a link mechanism.
• the trailing curtain solenoid is a drive source for the trailing curtain 68B, and is mechanically connected to the trailing curtain 68B via a link mechanism.
• Shutter actuator 70 is connected to controller 72 . Controller 72 controls shutter actuator 70 according to instructions from processor 34 .
• the front curtain solenoid generates power under the control of the controller 72, and selectively winds up and lowers the front curtain 68A by applying the generated power to the front curtain 68A.
• the trailing curtain solenoid generates power under the control of the controller 72, and selectively winds up and lowers the trailing curtain 68B by applying the generated power to the trailing curtain 68B.
• the opening/closing of the front curtain 68A and the opening/closing of the rear curtain 68B are controlled by the processor 34, thereby adjusting the amount of exposure to the image sensor 74.
• the shutter speed of the shutter 68 is determined by the time period during which the front curtain 68A and the rear curtain 68B are open.
• the shutter 68 may be a lens shutter.
• the shutter speed of the shutter 68 is specified, this is merely an example.
• the shutter speed of an electronic shutter (such as an electronic front curtain shutter or a fully electronic shutter) may be defined.
• a common computer 26 is used for the flying object 20 and the imaging device 60, but the computer 26 is connected to the first computer provided in the flying object 20 and the imaging device 60.
• the computer may be configured by a second computer installed in the computer.
• the computer 26 is mounted on the flying object 20, it may be mounted on the imaging device 60, or may be mounted on a transmitter or a base station.
• FIG. 4 shows an example of a mode in which the N-th imaging target area 3 and the N+1-th imaging target area 3 are imaged according to the flight position of the flight imaging device 10.
• N is a number indicating the order in which the images were taken, and is a natural number.
• the N-th imaging target region 3 and the N+1-th imaging target region 3 are regions imaged in the order of the N-th imaging target region 3 and the N+1-th imaging target region 3.
• the N-th imaging target region 3 and the N+1-th imaging target region 3 are regions aligned in the horizontal direction.
• Each imaging target area 3 has a first area 3A and a second area 3B.
• the first region 3A and the second region 3B are part of the imaging target region 3.
• the first region 3A is a region located in the front side of the imaging target region 3 in the flight direction of the aircraft 20, and the second region 3B is the rear region in the imaging target region 3 in the flight direction of the aircraft 20. It is an area located in The first region 3A of the N-th imaging target region 3 overlaps the second region 3B of the N+1-th imaging target region 3.
• the Nth imaging target area 3 is an example of a "first imaging target area” according to the technology of the present disclosure.
• the N+1-th imaging target area 3 is an example of a "second imaging target area” according to the technology of the present disclosure.
• the first area 3A of the Nth imaging target area 3 is an example of a "first area” according to the technology of the present disclosure.
• the second region 3B of the N+1-th imaging target region 3 is an example of a “second region” according to the technology of the present disclosure.
• the light source 48 is placed in front of the imaging device 60 in the flight direction of the flying object 20.
• the light source 48 irradiates each imaging target area 3 with light.
• the light source 48 is arranged at a position where the optical axis passes through each first region 3A (as an example, the center of each first region 3A) when each imaging target region 3 is imaged by the imaging device 60.
• the light source 48 is arranged at a position where it irradiates light along the normal direction of each first region 3A when each imaging target region 3 is imaged by the imaging device 60.
• the normal direction may be an average normal direction of the first region 3A.
• the orientation characteristics of the light source 48 are shown in FIG. (A) shows the light intensity distribution, and (B) shows the relationship between the light intensity and the distance from the center of the light.
• the light emitted by the light source 48 has a characteristic that the intensity decreases as it moves away from the center. Since the light source 48 has such an orientation characteristic, the intensity of light irradiated onto the first region 3A is higher than the intensity of light irradiated around the first region 3A.
• the size of the light source 48 is such that, for example, when the N-th image for synthesis 142 is obtained by capturing the N-th imaging target area 3, the size of the light source 48 is determined by The size is set such that an image 50 of the light source 48 (that is, an image of reflected light) can fit in the image area 142A corresponding to 3A.
• the image 50 of the light source 48 will be referred to as a "light source image 50.”
• a flight imaging program 90 is stored in the storage 36.
• the flight imaging program 90 is an example of a "program" according to the technology of the present disclosure.
• the processor 34 reads the flight imaging program 90 from the storage 36 and executes the read flight imaging program 90 on the RAM 38.
• the processor 34 performs flight imaging processing to image each imaging target area 3 while flying each flight route 4 according to the flight imaging program 90 executed on the RAM 38 .
• the processor 34 executes a lighting control section 92, a first arrival determination section 94, a first imaging control section 96, a first brightness acquisition section 98, a first exposure amount derivation section 100, and a first exposure amount derivation section 100 according to a flight imaging program 90.
• a lighting control section 92 executes a lighting control section 92, a first arrival determination section 94, a first imaging control section 96, a first brightness acquisition section 98, a first exposure amount derivation section 100, and a first exposure amount derivation section 100 according to a flight imaging program 90.
• 2 imaging control section 102, width acquisition section 104, imaging position correction section 106, second arrival determination section 108, third imaging control section 110, second brightness acquisition section 112, second exposure amount derivation section 114, fourth imaging is realized by operating as a control unit 116, an end determination unit 118, and a lights-out control unit 120.
• the flight imaging process is executed when the flight imaging device 10 starts flying from the starting end of each flight route
• the lighting control unit 92 outputs a lighting instruction signal to the light source 48 to turn on the light source 48. Thereby, the wall surface 2A is irradiated with light.
• the lighting control unit 92 causes the light source 48 to emit light until the flying object 20 reaches the N-th imaging position 5.
• the Nth imaging position 5 shown in FIG. 6 is, for example, the first imaging position 5.
• the intensity of the light emitted from the light source 48 is constant.
• the control by the lighting control unit 92 includes "control for causing the light source to irradiate light onto the first imaging target area" and “control for moving the moving body to a position where the first imaging target area is imaged by the imaging device" according to the technology of the present disclosure. This is an example of "control for causing a light source to emit light when moving.”
• Flight route information 122 indicating the flight route 4 is stored in the storage 36.
• the flight route information 122 includes imaging position information indicating the position of each imaging position 5 set on the flight route 4.
• the first arrival determination unit 94 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and/or the acceleration data input from the acceleration sensor 30. Then, the first arrival determination unit 94 determines whether the aircraft 20 has reached the N-th imaging position 5 based on the acquired position of the aircraft 20 and the N-th imaging position 5 indicated by the flight route information 122. Determine whether or not.
• FIG. 7 shows an example of a state in which the flying object 20 reaches the Nth imaging position 5.
• the flying object 20 reaches the Nth imaging position 5
• the light source 48 irradiates the Nth imaging target area 3 with light.
• the intensity of the light irradiated to the first region 3A of the Nth imaging target region 3 is equal to the intensity of the light irradiated to the periphery of the first region 3A of the Nth imaging target region 3. higher than the intensity of the light.
• the first imaging control section 96 When the first arrival determination section 94 determines that the flying object 20 has arrived at the N-th imaging position 5, the first imaging control section 96 outputs an imaging instruction signal to the image sensor 74 to The sensor 74 is caused to image the Nth imaging target area 3.
• Captured image data is obtained by capturing an image of the N-th imaging target region 3 by the image sensor 74 under the control of the first imaging control unit 96.
• the captured image data is image data indicating a captured image.
• the first imaging control unit 96 acquires a captured image corresponding to the Nth imaging target area 3 based on the captured image data.
• the captured image may be a brightness detection image for detecting brightness, or a display image such as a live view image displayed on a display device (not shown).
• the first brightness acquisition unit 98 acquires the brightness of the third area 3C other than the first area 3A of the Nth imaging target area 3 based on the captured image acquired by the first imaging control unit 96. do.
• the first brightness acquisition unit 98 may acquire the brightness of the center area of the Nth imaging target area 3 as the brightness of the third area 3C, or may acquire the brightness of the center area of the third area 3C.
• the brightness of the third area 3C may be acquired, or the brightness of the entire third area 3C may be acquired. Further, the brightness may be a representative value (for example, the highest value, the lowest value, or the center value) or an average value.
• the third area 3C is an example of a "third area" according to the technology of the present disclosure.
• the first exposure amount derivation unit 100 derives the exposure amount based on the brightness acquired by the first brightness acquisition unit 98.
• the exposure amount may be derived based on a table stored in the storage 36 or based on a relational expression stored in the storage 36.
• FIG. 7 shows a correspondence relationship between a graph showing pixel values of an N-th composite image 142 (see also FIG. 8), which will be described later, and the composite image 142.
• the graph showing the pixel values of the Nth synthesis image 142 has a shape corresponding to the graph showing the orientation characteristics of the light source 48 (see FIG. 4).
• the exposure amount is set as follows. That is, when the N-th image for synthesis 142 is obtained as described later, the pixel value of the image area 142A corresponding to the first area 3A of the N-th image for synthesis 142 is equal to or greater than the first predetermined pixel value.
• the exposure amount is set to .
• the first predetermined pixel value is set, for example, to a pixel value at which the pixel value of the image area 142A is saturated. Note that the first predetermined pixel value may be set to a pixel value at which the pixel values of at least a part of the image area 142A are saturated.
• the exposure amount is set to an amount such that the pixel value of the image area 142C corresponding to the third area 3C of the N-th composite image 142 is greater than or equal to the second predetermined pixel value and less than the first predetermined pixel value.
• the second predetermined pixel value is the brightness at which inspection and/or surveying can be performed (i.e., the brightness for inspection and/or surveying).
• the pixel value is set to correspond to the brightness suitable for By irradiating the imaging target area 3 with a light amount that saturates the pixel value of the image area 142A corresponding to the first area 3A, the third area 3C adjacent to the first area 3A is irradiated. The amount of light is ensured.
• FIG. 8 shows an example of a state in which the flying object 20 is temporarily stationary at the Nth imaging position 5.
• the second imaging control unit 102 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the N-th imaging target area 3. Further, when causing the image sensor 74 to image the N-th imaging target area 3, the second imaging control unit 102 sets the exposure amount of the light receiving surface 78A (see FIG. 3) of the image sensor 74 to the first exposure amount deriving unit. The exposure amount is adjusted to the value derived by 100.
• the exposure amount is adjusted, for example, by changing the exposure to the light receiving surface 78A.
• the second imaging control unit 102 controls the aperture actuator 66 to change the size of the aperture 64, and/or controls the shutter actuator 70 to change the shutter speed of the shutter 68. By performing control to change the exposure to the light receiving surface 78A.
• control in which the size of the aperture 64 is changed by the second imaging control unit 102, and control in which the shutter speed of the shutter 68 is changed by the second imaging control unit 102 (hereinafter referred to as "two types of control") Although an example is given in which control is performed, either one of the two types of control may be omitted.
• the N-th imaging target area 3 is imaged by the image sensor 74, thereby obtaining the N-th synthesis image data.
• the compositing image data is image data indicating the compositing image 142.
• the second imaging control unit 102 acquires the Nth synthesis image 142 corresponding to the Nth imaging target area 3 based on the synthesis image data.
• the light source image 50 is included in the image area 142A corresponding to the first area 3A of the Nth synthesis image 142.
• the Nth composite image 142 is stored in the storage 36.
• the flying object 20 resumes movement in the horizontal direction when the Nth composite image 142 is acquired.
• the pixel values of the image area 142A may vary as shown in FIG.
• the width Wa of the image area 142A (in other words, the position of the boundary between the image area 142A and the image area 142C) changes.
• the width acquisition unit 104 determines the width Wa of the image area 142A (that is, the pixel value is the first (width of an area that is greater than or equal to a predetermined pixel value). Then, the width acquisition unit 104 acquires the width W of the first area 3A by deriving the width W of the first area 3A corresponding to the width Wa of the image area 142A based on the width Wa of the image area 142A. .
• the width W of the first region 3A corresponds to the length of the first region 3A along the flight direction of the flying object 20.
• the flight direction is an example of the "movement direction” according to the technology of the present disclosure
• the width W of the first region 3A is an example of the "length of the first region” according to the technology of the present disclosure.
• FIG. 9 shows an example of the state before the flying object 20 reaches the N+1-th imaging position 5.
• the N+1-th imaging position 5 shown in FIG. 9 is, for example, the second imaging position 5.
• the imaging position correction unit 106 Based on the width W of the first area 3A acquired by the width acquisition unit 104, the imaging position correction unit 106 adjusts the N+1st imaging position 5 corresponding to the N+1th imaging target area 3, as shown by arrow C. Correct the position of.
• the position of the imaging position 5 is corrected, as shown by arrow D, the position of the N+1-th imaging target area 3 is corrected.
• the imaging position correction unit 106 changes the position of the N+1-th imaging position 5 to the first region 3A of the N-th imaging target region 3 (that is, having the width acquired by the width acquisition unit 104 The position is corrected so that the first region 3A) and the second region 3B of the N+1-th imaging target region 3 overlap.
• the second region 3B has the same width as the first region 3A.
• the second arrival determination unit 108 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and/or the acceleration data input from the acceleration sensor 30. Then, the second arrival determination unit 108 determines that the aircraft 20 is at the N+1-th imaging position 5 based on the acquired position of the aircraft 20 and the N+1-th imaging position 5 corrected by the imaging position correction unit 106. Determine whether it has been reached.
• FIG. 10 shows an example of a state in which the flying object 20 reaches the N+1-th imaging position 5.
• the flying object 20 reaches the N+1-th imaging position 5
• the light source 48 irradiates the N+1-th imaging target area 3 with light.
• the intensity of the light irradiated to the first region 3A of the N+1th imaging target region 3 is equal to the intensity of the light irradiated to the periphery of the first region 3A of the N+1th imaging target region 3. higher than the intensity of the light.
• the first area 3A of the N+1-th imaging target area 3 is an example of a "fourth area" according to the technology of the present disclosure.
• the third imaging control unit 110 When the arrival determination unit determines that the flying object 20 has reached the N+1-th imaging position 5, the third imaging control unit 110 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to receive an image.
• the N+1-th imaging target area 3 is imaged.
• Captured image data is obtained by capturing an image of the N+1-th imaging target region 3 by the image sensor 74 under the control of the third imaging control unit 110.
• the third imaging control unit 110 acquires a captured image corresponding to the N+1-th imaging target area 3 based on the captured image data.
• the second brightness acquisition unit 112 acquires the brightness of a third area 3C other than the first area 3A of the N+1th imaging target area 3 based on the captured image acquired by the third imaging control unit 110. do.
• the second brightness acquisition unit 112 may acquire the brightness of the center area of the N+1-th imaging target area 3 as the brightness of the third area 3C, or may acquire the brightness of the center area of the third area 3C.
• the brightness of the third area 3C may be acquired, or the brightness of the entire third area 3C may be acquired. Further, the brightness may be a representative value (for example, the highest value, the lowest value, or the center value) or an average value.
• the second exposure amount derivation unit 114 derives the exposure amount based on the brightness acquired by the second brightness acquisition unit 112.
• the exposure amount may be derived based on a table stored in the storage 36 or based on a relational expression stored in the storage 36.
• FIG. 10 shows the correspondence between the graph showing the pixel values of the N+1st image for synthesis 142 (see also FIG. 11), which will be described later, and the image for synthesis 142.
• the graph showing the pixel values of the N+1-th image for synthesis 142 has a shape corresponding to the graph showing the orientation characteristics of the light source 48 (see FIG. 4).
• the exposure amount is set as follows. That is, when the N+1st image for synthesis 142 is obtained as described later, the pixel value of the image area 142A corresponding to the first area 3A of the N+1st image for synthesis 142 is equal to or greater than the first predetermined pixel value.
• the exposure amount is set to .
• the first predetermined pixel value is set, for example, to a pixel value at which the pixel value of the image area 142A is saturated. Note that the first predetermined pixel value may be set to a pixel value at which the pixel values of at least a part of the image area 142A are saturated.
• the exposure amount is set to an amount such that the pixel value of the image area 142C corresponding to the third area 3C of the N+1st image for synthesis 142 is greater than or equal to the second predetermined pixel value and less than the first predetermined pixel value.
• the second predetermined pixel value is the brightness at which inspection and/or surveying can be performed (i.e., the brightness for inspection and/or surveying).
• the pixel value is set to correspond to the brightness suitable for By irradiating the imaging target area 3 with a light amount that saturates the pixel value of the image area 142A corresponding to the first area 3A, the third area 3C adjacent to the first area 3A is irradiated. The amount of light is ensured.
• FIG. 11 shows an example of a state in which the flying object 20 is temporarily stationary at the N+1-th imaging position 5.
• the fourth imaging control unit 116 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the N+1-th imaging target region 3. Further, when causing the image sensor 74 to image the N+1-th imaging target region 3, the fourth imaging control unit 116 controls the exposure amount of the light-receiving surface 78A (see FIG. 3) to be derived by the second exposure amount deriving unit 114. Adjust the exposure amount to the desired value.
• the exposure amount is adjusted, for example, by changing the exposure to the light receiving surface 78A.
• the fourth imaging control unit 116 controls the aperture actuator 66 to change the size of the aperture 64, and/or controls the shutter actuator 70 to change the shutter speed of the shutter 68. By performing control to change the exposure to the light receiving surface 78A.
• control in which the size of the aperture 64 is changed by the fourth imaging control unit 116, and control in which the shutter speed of the shutter 68 is changed by the fourth imaging control unit 116 (hereinafter referred to as "two types of control") Although an example is given in which control is performed, either one of the two types of control may be omitted.
• the N+1st imaging target area 3 is captured by the image sensor 74, thereby obtaining the N+1st image data for synthesis.
• the fourth imaging control unit 116 acquires the N+1st image for synthesis 142 corresponding to the N+1th imaging target area 3 based on the image data for synthesis.
• the image area 142A corresponding to the first area 3A of the N+1st image for synthesis 142 includes the light source image 50
• the third area 3C other than the first area 3A of the N+1st image for synthesis 142 includes the light source image 50
• the image area 142C corresponding to 142C does not include the light source image 50.
• the third area 3C other than the first area 3A includes the second area 3B
• the image area 142C other than the image area 142A includes an image area 142B corresponding to the second area 3B.
• the N+1-th composite image 142 is stored in the storage 36.
• the flying object 20 resumes movement in the horizontal direction when the N+1-th composite image 142 is acquired.
• the termination determination unit 118 determines whether a condition for terminating the flight imaging process (hereinafter referred to as "termination condition") is satisfied.
• the termination condition is that the number of multiple images 142 for synthesis stored in the storage 36 has reached the number of multiple imaging positions 5 set on the flight route 4, or the condition that the number of images 142 for synthesis stored in the storage 36 has reached the number of multiple imaging positions 5 set on the flight route 4, or that the flight imaging device is For example, the condition that an instruction to end the flight imaging process has been given to 10 is given.
• the N+1st image for synthesis 142 is treated as the Nth image for synthesis 142. Then, in order to obtain a new N+1-th composite image 142, the processes from the width acquisition unit 104 to the fourth imaging control unit 116 are executed again.
• the lights-off control unit 120 When the termination determining unit 118 determines that the termination condition is satisfied, the lights-off control unit 120 outputs a lights-out instruction signal to the light source 48, thereby extinguishing the light source 48.
• the flight imaging process is executed by the flight imaging device 10, the flight imaging process is performed by a transmitter (not shown) or a base station communicably connected to the flight imaging device 10. (not shown).
• a composite image generation program 130 is stored in the storage 36.
• the composite image generation program 130 is an example of a "program" according to the technology of the present disclosure.
• the processor 34 reads the composite image generation program 130 from the storage 36 and executes the read composite image generation program 130 on the RAM 38.
• the processor 34 performs a composite image generation process for imaging each imaging target area 3 while flying each flight route 4 according to a composite image generation program 130 executed on the RAM 38 .
• the composite image generation process is realized by the processor 34 operating as the image acquisition section 132 and the image composition section 134 according to the flight imaging program 90.
• the composite image generation process may be executed each time each composite image 142 from the second frame onwards is obtained, or may be executed after a plurality of composite images 142 are obtained for the wall surface 2A.
• the composite image generation process will be described below, taking as an example a case where the composite image 140 is generated by combining the N-th composite image 142 and the N+1-th composite image 142.
• the image acquisition unit 132 acquires the N-th composite image 142 and the N+1-th composite image 142 stored in the storage 36.
• the image area 142A of the N-th image for synthesis 142 includes the light source image 50, and the image area 142B of the N+1-th image for synthesis 142 does not include the light source image 50.
• the image composition unit 134 generates a composite image 140 by combining the N-th composite image 142 and the N+1-th composite image 142.
• the image composition unit 134 uses the pixel value of the image area 142B of the N+1-th composition image 142 as the pixel value of the overlap image region 144 of the composite image 140.
• a composite image 140 in which the light source image 50 is not included in the overlapping image area 144 is obtained.
• the overlapping image area 144 is an area corresponding to the first area 3A and the second area 3B (see FIG. 12) that overlap with each other.
• the composite image generation process is executed by the flight imaging device 10
• the composite image generation process is performed by an external device (not shown) communicatively connected to the flight imaging device 10. May be executed.
• the last composite image 142 included in the composite image 140 includes the light source image 50. Therefore, for example, when a composite image 140 is generated based on three or more frames of composite images 142, the image area 142A of the last composite image 142 included in the composite image 140 is deleted from the composite image 140. may be done. Furthermore, the first region 3A of the last imaging target region 3 may be a region other than the inspection target region.
• the Nth synthesis image 142 is an example of a "first captured image” according to the technology of the present disclosure.
• the N+1st image for synthesis 142 is an example of a "second captured image” according to the technology of the present disclosure.
• the composite image 140 is an example of a "composite image” according to the technology of the present disclosure.
• FIG. 15 shows an example of the flow of flight imaging processing according to the first embodiment.
• step ST10 the lighting control section 92 turns on the light source 48 (see FIG. 6). Thereby, the wall surface 2A is irradiated with light. After the process of step ST10 is executed, the flight imaging process moves to step ST12.
• step ST12 the first arrival determination unit 94 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and/or the acceleration data input from the acceleration sensor 30. Then, the first arrival determination unit 94 determines whether the aircraft 20 has reached the N-th imaging position 5 based on the acquired position of the aircraft 20 and the N-th imaging position 5 indicated by the flight route information 122. (See FIG. 6). In step ST12, when the flying object 20 reaches the Nth imaging position 5, the determination is affirmative and the flight imaging process moves to step ST14. In step ST12, if the flying object 20 has not reached the Nth imaging position 5, the determination is negative and the flight imaging process executes the process of step ST12 again.
• step ST14 the first imaging control unit 96 causes the image sensor 74 to image the N-th imaging target region 3 (see FIG. 7). As a result, a captured image corresponding to the Nth imaging target area 3 is obtained.
• step ST14 the flight imaging process moves to step ST16.
• step ST16 the first brightness acquisition unit 98 acquires the brightness of the third area 3C other than the first area 3A of the Nth imaging target area 3 based on the captured image acquired in step ST14. (See Figure 7). After the process of step ST16 is executed, the flight imaging process moves to step ST18.
• step ST18 the first exposure amount deriving unit 100 derives the exposure amount based on the brightness acquired in step ST16 (see FIG. 7). After the process of step ST18 is executed, the flight imaging process moves to step ST20.
• step ST20 the second imaging control unit 102 causes the image sensor 74 to image the N-th imaging target area 3 (see FIG. 8).
• the second imaging control unit 102 controls the aperture actuator 66 and/or the shutter actuator 70 to control the exposure amount of the light receiving surface 78A.
• the exposure amount is adjusted to the amount derived in step ST18.
• a composition image 142 corresponding to the Nth imaging target region 3 is obtained.
• step ST22 the width acquisition unit 104 calculates the width Wa of the image area 142A based on the pixel value of the image area 142A corresponding to the first area 3A of the N-th synthesis image 142 acquired in step ST20. Identify. Then, the width acquisition unit 104 acquires the width W of the first area 3A by deriving the width W of the first area 3A corresponding to the width Wa of the image area 142A based on the width Wa of the image area 142A. (See Figure 8). After the process of step ST22 is executed, the flight imaging process moves to step ST24.
• step ST24 the imaging position correction unit 106 corrects the position of the imaging position 5 corresponding to the N+1-th imaging target area 3 based on the width W of the first area 3A acquired by the width acquisition unit 104 (Fig. 9). After the process of step ST24 is executed, the flight imaging process moves to step ST26.
• step ST26 the second arrival determination unit 108 acquires the position of the flying object 20 based on the positioning data input from the positioning unit 28 and/or the acceleration data input from the acceleration sensor 30. Then, the second arrival determination unit 108 determines whether the flying object 20 has reached the N+1-th imaging position 5 based on the acquired position of the flying object 20 and the N+1-th imaging position 5 corrected in step ST24. It is determined whether or not (see FIG. 9). In step ST26, when the flying object 20 reaches the N+1-th imaging position 5, the determination is affirmative and the flight imaging process moves to step ST28. In step ST26, if the flying object 20 has not reached the N+1-th imaging position 5, the determination is negative and the flight imaging process executes the process of step ST26 again.
• step ST28 the third imaging control unit 110 causes the image sensor 74 to image the N+1-th imaging target area 3 (see FIG. 10). As a result, a captured image corresponding to the N+1-th imaging target area 3 is obtained.
• step ST30 the flight imaging process moves to step ST30.
• step ST30 the second brightness acquisition unit 112 acquires the brightness of the third region 3C other than the first region 3A of the N+1-th imaging target region 3 based on the captured image acquired in step ST28. (See Figure 10). After the process of step ST30 is executed, the flight imaging process moves to step ST32.
• step ST32 the second exposure amount deriving unit 114 derives the exposure amount based on the brightness obtained in step ST30 (see FIG. 10). After the process of step ST32 is executed, the flight imaging process moves to step ST34.
• step ST34 the fourth imaging control unit 116 causes the image sensor 74 to image the N+1-th imaging target area 3 (see FIG. 11). Further, when the image sensor 74 is to image the N+1-th imaging target region 3, the fourth imaging control unit 116 controls the aperture actuator 66 and/or the shutter actuator 70 to control the exposure amount of the light receiving surface 78A. The exposure amount is adjusted to the amount derived in step ST32. The image for synthesis 142 corresponding to the N+1-th imaging target region 3 is obtained by imaging the N+1-th imaging target region 3 with the exposure amount adjusted.
• the flight imaging process moves to step ST36.
• step ST36 the termination determination unit 118 determines whether the termination condition for terminating the flight imaging process is satisfied (see FIG. 12). In step ST36, if the end condition is not satisfied, the determination is negative and the flight imaging process moves to step ST38. In step ST36, if the termination condition is satisfied, the determination is affirmative and the flight imaging process moves to step ST40.
• step ST38 the processor 34 treats the N+1st image for synthesis 142 as the Nth image for synthesis 142. After the process of step ST38 is executed, the flight imaging process moves to step ST22.
• step ST40 the lights-off control section 120 turns off the light source 48 (see FIG. 12). After the process of step ST40 is executed, the flight imaging process ends.
• FIG. 16 shows an example of the flow of composite image generation processing according to the first embodiment.
• step ST50 the image acquisition unit 132 acquires the N-th composite image 142 and the N+1-th composite image 142 stored in the storage 36 (FIG. reference). After the process of step ST50 is executed, the composite image generation process moves to step ST52.
• step ST52 the image composition unit 134 generates a composite image 140 by combining the N-th composite image 142 and the N+1-th composite image 142 (see FIG. 14).
• the image composition unit 134 uses the pixel value of the image area 142B of the N+1-th composition image 142 as the pixel value of the overlap image region 144 of the composite image 140.
• a composite image 140 in which the light source image 50 is not included in the overlapping image area 144 is obtained.
• the composite image generation process moves to step ST54.
• step ST54 the processor 34 determines whether a condition for terminating the composite image generation process (hereinafter referred to as "termination condition") is satisfied.
• the termination condition is that all of the plurality of composite images 142 stored in the storage 36 have been composited, or the user or the like has given an instruction to the flight imaging device 10 to terminate the composite image generation process. Examples include conditions such as: In step ST54, if the end condition is not satisfied, the determination is negative and the composite image generation process moves to step ST50. In step ST54, if the termination condition is satisfied, the determination is affirmative and the composite image generation process is terminated.
• the control method described as the operation of the flight imaging device 10 described above is an example of a "control method" according to the technology of the present disclosure.
• the composite image 140 is generated by combining the N-th composite image 142 and the N+1-th composite image 142 (see FIG. 14).
• the pixel value of the image area 142B of the N+1-th composite image 142 is used as the pixel value of the overlap image area 144 of the composite image 140. Therefore, it is possible to avoid using the pixel value of the image area 142A (that is, the image area 142A including the light source image 50) of the Nth synthesis image 142 as the pixel value of the overlap image area 144. .
• the light source image 50 fits in the image area 142A of the N-th image for synthesis 142 (see FIG. 4). Therefore, by avoiding using the pixel value of the image area 142A of the Nth image for synthesis 142 as the pixel value of the overlap image area 144, a synthesized image 140 that does not include the light source image 50 is obtained. be able to.
• the intensity of the light irradiated to the first region 3A of the N-th imaging target region 3 is higher than the intensity of the light irradiated to the periphery of the first region 3A (see FIG. 4). . Therefore, while increasing the intensity of the light in the first area 3A corresponding to the image area 142A of the Nth synthesis image 142, the intensity of the light around the first area 3A (that is, the intensity of the light in the third area 3C) is increased. strength) can be ensured. As a result, the brightness of the third area 3C of the Nth imaging target area 3 can be ensured, so that the appropriate brightness of the entire composite image 140 can be secured as the Nth composite image 142. Contributing images can be obtained.
• the exposure amount of the light receiving surface 78A is adjusted based on the brightness of the third area 3C of the Nth imaging target area 3. (See Figures 7 and 8). Therefore, by adjusting the exposure amount according to the brightness of the third area 3C, it is possible to ensure the brightness of the image area 142C corresponding to the third area 3C in the Nth compositing image 142. .
• control to change the size of the aperture 64 and/or control to change the shutter speed by performing control to change the size of the aperture 64 and/or control to change the shutter speed, the exposure to the light receiving surface 78A is changed, and thereby the exposure amount is adjusted. (See Figure 8). Therefore, compared to the case where special control other than control to change the size of the aperture 64 and/or control to change the shutter speed is performed, it is possible to adjust the exposure amount by simple control. can.
• the exposure amount is set to an amount such that the pixel value of the image area 142A of the Nth image for synthesis 142 is equal to or greater than the first predetermined pixel value (see FIG. 7). Therefore, by avoiding using the pixel value of the image area 142A of the Nth image for synthesis 142 as the pixel value of the overlap image area 144, the pixel value of the overlap image area 144 is It is possible to avoid the pixel value from exceeding the predetermined pixel value.
• the first predetermined pixel value is set to a pixel value at which the pixel values of at least a part of the image area 142A are saturated (see FIG. 7). Therefore, by avoiding using the pixel value of the image area 142A of the N-th synthesis image 142 as the pixel value of the overlap image area 144, the pixel value of at least a part of the area is saturated. Generating the composite image 140 can be avoided.
• light is used that has such an intensity that the pixel values of at least a part of the image area 142A are saturated.
• the exposure time can be shortened compared to the case where light having an intensity that does not saturate the pixel values is used, so that image blur can be suppressed.
• the exposure amount is set to an amount such that the pixel value of the image area 142C of the Nth image for synthesis 142 is greater than or equal to the second predetermined pixel value and less than the first predetermined pixel value (see FIG. 7). Therefore, it is possible to ensure the brightness of the image area 142C of the Nth image for synthesis 142.
• the width Wa of the image area 142A is specified based on the pixel value of the image area 142A of the Nth image for synthesis 142, and corresponds to the width Wa of the image area 142A based on the width Wa of the image area 142A.
• the width W of the first region 3A is derived (see FIG. 8). Then, the position of the imaging position 5 corresponding to the N+1-th imaging target region 3 is corrected based on the width W of the first region 3A (see FIG. 9). Therefore, even if the width W of the first area 3A corresponding to the width Wa of the image area 142A changes as the pixel value of the image area 142A changes, the first area 3A and the second area 3B overlap. You can change the amount of overlap.
• the light source 48 is placed in front of the imaging device 60 in the flight direction of the flight imaging device 10 (see FIG. 4). Therefore, when the composite image 140 is generated by combining the N-th composite image 142 and the N+1-th composite image 142, it is assumed that the light source image 50 is included in the N-th composite image 142. can be avoided.
• the light is irradiated along the normal direction of the first region 3A (see FIG. 4).
• the intensity of the light irradiated to the first region 3A of the N-th imaging target region 3 can be made higher than the intensity of the light irradiated to the periphery of the first region 3A. .
• the intensity of the light irradiated onto the first region 3A of the N+1-th imaging target region 3 is higher than the intensity of the light irradiated around the first region 3A (see FIG. 10). .
• the intensity of the light around the first area 3A (that is, the light in the third area 3C) is increased. strength) can be ensured.
• the brightness of the third region 3C of the N+1st imaging target region 3 can be ensured, so that the appropriate brightness of the entire composite image 140 can be secured as the N+1st composite image 142. Contributing images can be obtained.
• the flight imaging device 10 moves to each imaging position 5 with the light source 48 irradiating light. Therefore, it is not necessary to control the light source 48 to turn on and off every time the imaging position 5 is reached.
• FIG. 17 shows an example of a state in which the flying object 20 reaches the Nth imaging position 5.
• the flight imaging process is changed as follows compared to the first embodiment. That is, in the second embodiment, the processing by the width acquisition unit 104 (see FIG. 8) and the processing by the imaging position correction unit 106 (see FIG. 9) are omitted. Further, in the second embodiment, the amount of overlap OL by which the first region 3A and the second region 3B overlap is fixed to a predetermined amount of overlap.
• the exposure amount derived by the first exposure amount derivation unit 100 is the width Wa of the image area 142A of the N-th synthesis image 142 (that is, the width of the area where the pixel value is equal to or greater than the first predetermined pixel value), which will be described later.
• the width W of the first region 3A corresponding to the width W is set to an exposure amount that matches the overlap amount OL.
• the processor 34 (see FIG. 5) operates as the light source control unit 124.
• the light source control unit 124 changes the intensity of light emitted from the light source 48 by controlling the light source 48. Specifically, the light source control unit 124 changes the intensity of light based on the exposure amount derived by the first exposure amount derivation unit 100.
• the second imaging control unit 102 causes the image sensor 74 to image the N-th imaging target region 3 by outputting an imaging instruction signal to the image sensor 74. Further, when causing the image sensor 74 to image the N-th imaging target region 3, the second imaging control unit 102 controls the light receiving surface 78A (see FIG. 3) based on the exposure amount derived by the first exposure amount derivation unit 100. ) to adjust the exposure amount.
• the exposure amount is adjusted, for example, by changing the exposure to the light receiving surface 78A.
• the second imaging control unit 102 controls the aperture actuator 66 to change the size of the aperture 64, and/or controls the shutter actuator 70 to change the shutter speed of the shutter 68. By performing control to change the exposure to the light receiving surface 78A.
• the light source control unit 124 controls the intensity of light to be changed
• the second imaging control unit 102 controls the size of the aperture 64
• the second imaging control unit 102 changes the shutter speed of the shutter 68.
• the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the first exposure amount deriving section 100.
• the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the first exposure amount deriving section 100 by performing three types of control.
• the width W of the first area 3A corresponding to the width Wa of the image area 142A (that is, the width of the area whose pixel value is greater than or equal to the first predetermined pixel value) of the N-th image for synthesis 142 is increased by the amount of overlap. Since the exposure amount of the light receiving surface 78A is set to the exposure amount that matches OL, the overlap amount OL can be fixed to a predetermined overlap amount.
• the overlap amount OL is fixed to the predetermined overlap amount, the overlap amount OL is changed to a larger overlap amount than the predetermined overlap amount, so that multiple imaging target regions 3 The efficiency of imaging can be improved.
• the exposure amount of the light receiving surface 78A may be set to the exposure amount derived by the first exposure amount deriving section 100 by performing any one or two of the three types of control.
• the light source 48 may be configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position. Further, the light source 48 is configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position and the light intensity. Even in this case, the exposure amount of the light receiving surface 78A can be set to the exposure amount derived by the first exposure amount deriving section 100.
• the flight imaging device 10 is changed from the second embodiment as follows. That is, the light source 48 is arranged on the rear side of the imaging device 60 in the flight direction of the flying object 20 .
• the light source 48 irradiates each imaging target area 3 with light.
• the light source 48 is arranged at a position where the optical axis passes through each second region 3B (as an example, the center of each second region 3B) when each imaging target region 3 is imaged by the imaging device 60.
• the light source 48 is arranged at a position where it irradiates light along the normal direction of each second region 3B when each imaging target region 3 is imaged by the imaging device 60.
• the normal direction may be an average normal direction of the second region 3B.
• the size of the light source 48 is such that, for example, when the N-th image for synthesis 142 is obtained by capturing the N-th imaging target area 3, the size of the second area of the N-th image for synthesis 142 is determined.
• the size is set such that the light source image 50 (that is, the image of reflected light) fits in the image area 142B corresponding to 3B.
• the N+1-th imaging target area 3 is an example of a "first imaging target area” according to the technology of the present disclosure.
• the Nth imaging target area 3 is an example of a "second imaging target area” according to the technology of the present disclosure.
• the second region 3B of the N+1-th imaging target region 3 is an example of a "first region” according to the technology of the present disclosure.
• the first region 3A of the N-th imaging target region 3 is an example of a “second region” according to the technology of the present disclosure.
• FIG. 19 shows an example of a state in which the flying object 20 reaches the N+1-th imaging position 5.
• the flight imaging process is modified as follows compared to the first embodiment. That is, in the third embodiment, the processing by the width acquisition unit 104 (see FIG. 8) and the processing by the imaging position correction unit 106 (see FIG. 9) are omitted. Further, in the third embodiment, the amount of overlap OL by which the first region 3A and the second region 3B overlap is fixed to a predetermined amount of overlap. The exposure amount is derived based on the brightness of the fourth region 3D other than the second region 3B of the N+1-th imaging target region 3.
• the fourth area 3D is an example of a "third area" according to the technology of the present disclosure.
• the exposure amount derived by the second exposure amount derivation unit 114 corresponds to the width Wb of the image area 142B of the N+1st image for synthesis 142 (that is, the width of the area where the pixel value is equal to or greater than the first predetermined pixel value).
• the width of the second region 3B is set to an exposure amount that matches the overlap amount OL.
• the processor 34 (see FIG. 5) operates as the light source control unit 124.
• the light source control unit 124 changes the intensity of light emitted from the light source 48 by controlling the light source 48. Specifically, the light source control unit 124 changes the intensity of light based on the exposure amount derived by the second exposure amount derivation unit 114.
• the fourth imaging control unit 116 outputs an imaging instruction signal to the image sensor 74, thereby causing the image sensor 74 to image the N+1-th imaging target area 3. Further, when causing the image sensor 74 to image the N+1-th imaging target region 3, the fourth imaging control unit 116 controls the light receiving surface 78A (see FIG. 3) based on the exposure amount derived by the second exposure amount derivation unit 114. ) to adjust the exposure amount.
• the exposure amount is adjusted, for example, by changing the exposure to the light receiving surface 78A.
• the fourth imaging control unit 116 controls the aperture actuator 66 to change the size of the aperture 64, and/or controls the shutter actuator 70 to change the shutter speed of the shutter 68. By performing control to change the exposure to the light receiving surface 78A.
• the light source control unit 124 controls the intensity of light to be changed
• the second imaging control unit 102 controls the size of the aperture 64
• the second imaging control unit 102 changes the shutter speed of the shutter 68.
• the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the second exposure amount deriving section 114.
• the composite image generation process is changed from the first embodiment as follows. That is, in the third embodiment, the image acquisition unit 132 acquires the N-th composition image 142 and the N+1-th composition image 142 stored in the storage 36.
• the image area 142B of the N+1st image for synthesis 142 includes the light source image 50, and the image area 142A of the Nth image for synthesis 142 does not include the light source image 50.
• the image composition unit 134 generates a composite image 140 by combining the N-th composite image 142 and the N+1-th composite image 142.
• the image composition unit 134 uses the pixel value of the image area 142A of the N-th composition image 142 as the pixel value of the overlap image region 144 of the composite image 140. As a result, a composite image 140 in which the light source image 50 is not included in the overlapping image area 144 is obtained.
• the exposure amount of the light receiving surface 78A is set to the exposure amount derived by the second exposure amount deriving section 114.
• the width W of the second area 3B corresponding to the width Wb of the image area 142B (that is, the width of the area whose pixel value is greater than or equal to the first predetermined pixel value) of the N+1st image for synthesis 142 is changed by the amount of overlap. Since the exposure amount of the light receiving surface 78A is set to the exposure amount that matches OL, the overlap amount OL can be fixed to a predetermined overlap amount.
• the overlap amount OL is fixed to the predetermined overlap amount, the overlap amount OL is changed to a larger overlap amount than the predetermined overlap amount, so that multiple imaging target regions 3 The efficiency of imaging can be improved.
• the light source 48 is arranged on the rear side of the imaging device 60 in the flight direction of the flight imaging device 10. Therefore, when the composite image 140 is generated by combining the N-th composite image 142 and the N+1-th composite image 142, it is assumed that the light source image 50 is included in the N+1-th composite image 142. can be avoided.
• the exposure amount of the light receiving surface 78A may be set to the exposure amount derived by the second exposure amount deriving section 114 by performing any one or two of the three types of control.
• the light source 48 may be configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position. Further, the light source 48 is configured to be able to change the light irradiation position, and the exposure amount may be adjusted by changing the light irradiation position and the light intensity. In this case as well, the exposure amount of the light receiving surface 78A can be set to the exposure amount derived by the second exposure amount deriving section 114.
• the first composite image 142 included in the composite image 140 includes the light source image 50. Therefore, for example, when a composite image 140 is generated based on three or more frames of composite images 142, the image area 142B of the first composite image 142 included in the composite image 140 is May be deleted. Further, the second region 3B of the first imaging target region 3 may be a region other than the inspection target region.
• the N+1st image for synthesis 142 is an example of a "first captured image” according to the technology of the present disclosure.
• the Nth synthesis image 142 is an example of a "second captured image” according to the technology of the present disclosure.
• the plurality of flight routes 4 are changed from the first embodiment as follows. That is, the plurality of flight routes 4 include two types of flight routes 4 having different directions.
• the flight imaging device 10 is changed from the first embodiment as follows. That is, the flight imaging device 10 includes a first light source 48A and a second light source 48B.
• the first light source 48A and the second light source 48B are an example of a "light source” according to the technology of the present disclosure.
• the first light source 48A is disposed on the first side (for example, on the right side) of the flying object 20 with respect to the imaging device 60, and the second light source 48B is located on the first side of the flying object 20 with respect to the imaging device 60. It is arranged on the second side (as an example, the left side) in the flight direction.
• the first light source 48A and the second light source 48B have the same configuration as the light source 48 according to the first embodiment.
• the second light source 48B is arranged at a position symmetrical to the first light source 48A in the flight direction of the flying object 20.
• the processor 34 turns on the first light source 48A disposed on the first side with respect to the imaging device 60, so that the flying object 20 flies to the first side.
• the second light source 48B arranged on the second side with respect to the imaging device 60 is turned on.
• the flight imaging process is executed when the flight imaging device 10 starts flying from the starting point of each flight route 4.
• the plurality of imaging target regions 3 are sequentially imaged by the imaging device 60, so that the plurality of flight routes 4 include two types of flight routes 4 having different directions.
• a composite image 142 is obtained. Therefore, the composite image 140 can be generated by combining the plurality of composite images 142.
• the fourth embodiment may be combined with the third embodiment.
• the processor 34 turns on the second light source 48B arranged on the second side with respect to the imaging device 60, so that the flying object 20 flies to the second side.
• the second light source 48B disposed on the first side with respect to the imaging device 60 may be turned on. Even in this case, a plurality of images for synthesis 142 are obtained by sequentially capturing images of the plurality of imaging target regions 3 by the imaging device 60. Therefore, the composite image 140 can be generated by combining the plurality of composite images 142.
• the position of the light source 48 is changed from the first embodiment as follows. That is, the light source 48 is arranged above the imaging device 60. The light source 48 irradiates each imaging target area 3 with light. Each imaging target area 3 has a first area 3A and a second area 3B. The first region 3A and the second region 3B are part of the imaging target region 3. The first region 3A is an upper region of the imaging target region 3, and the second region 3B is a lower region of the imaging target region 3.
• the light source 48 is arranged at a position where the optical axis passes through each first region 3A (as an example, the center of the first region 3A) when each imaging target region 3 is imaged by the imaging device 60.
• the light source 48 is arranged at a position where it irradiates light along the normal direction of each first region 3A when each imaging target region 3 is imaged by the imaging device 60.
• the normal direction may be an average normal direction of the first region 3A.
• the intensity of the light irradiated onto the first region 3A is higher than the intensity of the light irradiated around the first region 3A.
• the size of the light source 48 is such that when the image for synthesis 142 is obtained by capturing the image capturing target area 3, the light source image is placed in the image area 142A corresponding to the first area 3A of the image for synthesis 142. 50 (that is, an image of reflected light).
• the composite image generation process is changed from the first embodiment as follows. That is, when the flight imaging device 10 captures an image while flying the Nth flight route 4, the image composition unit 134 composes the plurality of composition images 142 obtained for the Nth flight route 4, and A second composite image 150 is generated.
• the flight imaging device 10 When the flight imaging device 10 performs imaging while flying the N-th flight route 4, the flight imaging device 10 moves to the N+1-th flight route 4 one above the N-th flight route 4, and moves to the N+1-th flight route 4 while flying the N-th flight route 4. Capture images while flying.
• the image composition unit 134 synthesizes the plurality of composition images 142 obtained for the N+1th flight route 4, and A composite image 150 is generated.
• the image composition unit 134 generates a composite image 140 by combining the N-th composite image 150 and the N+1-th composite image 150.
• the image synthesis unit 134 uses the pixel value of the image area 142B of the N+1-th composite image 150 as the pixel value of the overlap image area 154 of the composite image 140. As a result, it is possible to obtain a composite image 140 in which the light source image 50 is not included in the overlapping image area 154.
• the N-th imaging target area 3 is an example of a "first imaging target area” according to the technology of the present disclosure.
• the N+1-th imaging target area 3 is an example of a "second imaging target area” according to the technology of the present disclosure.
• the first area 3A is an example of a "first area” according to the technology of the present disclosure.
• the second area 3B is an example of a "second area” according to the technology of the present disclosure.
• the Nth composite image 150 is an example of a "first captured image” according to the technology of the present disclosure.
• the N+1-th composite image 150 is an example of a "second captured image” according to the technology of the present disclosure.
• the position of the light source 48 is changed from the third embodiment as follows. That is, the light source 48 is arranged below the imaging device 60. The light source 48 irradiates each imaging target area 3 with light. Each imaging target area 3 has a first area 3A and a second area 3B. The first region 3A and the second region 3B are part of the imaging target region 3. The first region 3A is an upper region of the imaging target region 3, and the second region 3B is a lower region of the imaging target region 3.
• the light source 48 is arranged at a position where the optical axis passes through each second region 3B (as an example, the center of the second region 3B) when each imaging target region 3 is imaged by the imaging device 60.
• the light source 48 is arranged at a position where it irradiates light along the normal direction of each second region 3B when each imaging target region 3 is imaged by the imaging device 60.
• the normal direction may be an average normal direction of the second region 3B.
• the intensity of the light irradiated onto the second region 3B is higher than the intensity of the light irradiated around the second region 3B.
• the size of the light source 48 is such that when the image for synthesis 142 is obtained by capturing the image capture target area 3, the light source image is placed in the image area 142B corresponding to the second area 3B of the image for synthesis 142. 50 (that is, an image of reflected light).
• the composite image generation process is changed from the third embodiment as follows. That is, when the flight imaging device 10 captures an image while flying the Nth flight route 4, the image composition unit 134 composes the plurality of composition images 142 obtained for the Nth flight route 4, and A second composite image 150 is generated.
• the flight imaging device 10 When the flight imaging device 10 performs imaging while flying the N-th flight route 4, the flight imaging device 10 moves to the N+1-th flight route 4 one above the N-th flight route 4, and moves to the N+1-th flight route 4 while flying the N-th flight route 4. Capture images while flying.
• the image composition unit 134 synthesizes the plurality of composition images 142 obtained for the N+1th flight route 4, and A composite image 150 is generated.
• the image composition unit 134 generates a composite image 140 by combining the N-th composite image 150 and the N+1-th composite image 150.
• the image synthesis unit 134 uses the pixel value of the image area 142A of the Nth composite image 150 as the pixel value of the overlap image area 154 of the composite image 140. As a result, it is possible to obtain a composite image 140 in which the light source image 50 is not included in the overlapping image area 154.
• the N+1-th imaging target area 3 is an example of a "first imaging target area” according to the technology of the present disclosure.
• the Nth imaging target area 3 is an example of a "second imaging target area” according to the technology of the present disclosure.
• the second region 3B of the N+1-th imaging target region 3 is an example of a "first region” according to the technology of the present disclosure.
• the first region 3A of the N-th imaging target region 3 is an example of a “second region” according to the technology of the present disclosure.
• the N+1-th composite image 150 is an example of a "first captured image” according to the technology of the present disclosure.
• the Nth composite image 150 is an example of a "second captured image” according to the technology of the present disclosure.
• the lighting control section 92 turns on the light source 48 when the flight imaging device 10 starts flight on the flight route 4, and the lights-off control section 120 turns on the light source 48 when the flight imaging device 10 starts flight on the flight route 4.
• the light source 48 is turned off.
• the lighting control unit 92 turns on the light source 48 when the flight imaging device 10 reaches each imaging position 5 on the flight route 4, and the lights-off control unit 120 controls the lighting control unit 92 to turn on the light source 48 when the flight imaging device 10 reaches each imaging position 5.
• the light source 48 may be turned off.
• the control by the lighting control unit 92 in this case is an example of "control for causing the light source to emit light when the moving object moves to a position where the first imaging target area is imaged by the imaging device" according to the technology of the present disclosure. It is.
• the size of the light source 48 is such that, for example, when the N-th image for synthesis 142 is obtained by capturing the N-th imaging target area 3, the size of the light source 48 is The size is set so that the light source image 50 (that is, the image of reflected light) can fit in the image area 142A.
• control for the light source image 50 to fit in the image area 142A includes control for increasing or decreasing the intensity of the light emitted from the light source 48, control for changing the position of the light source 48, control for changing the angle of the light source 48, or control for flying with the wall surface 2A. Examples include control to change the distance to the imaging device 10. Even in this case, since the pixel value of the image area 142A of the N-th image for synthesis 142 is avoided as the pixel value of the overlap image area 144, the synthesis that does not include the light source image 50 An image 140 can be obtained.
• the brightness of the imaging target region 3 is acquired based on the captured image obtained by imaging by the imaging device 60.
• the brightness of the imaging target area 3 may be acquired based on a captured image obtained by capturing an image of the wall surface 2A with an overhead camera (not shown).
• the flight imaging device 10 is equipped with a sensor (not shown) that detects the brightness of the imaging target region 3 separately from the imaging device 60, and the brightness of the imaging target region 3 is acquired based on the detection result of the sensor. may be done.
• the composite image 142 is acquired while the flight imaging device 10 is temporarily stopped at each imaging position 5.
• the flying imaging device 10 may acquire the composite image 142 while passing through each imaging device 60.
• the flying object 20 is illustrated as an example of a moving object, but any moving object may be used as long as it moves on a moving route.
• the moving object include a car, motorcycle, bicycle, trolley, gondola, airplane, flying object, or ship.
• the flight route 4 extends along the horizontal direction, but it may extend in a direction other than the horizontal direction. Further, the direction of the flight route 4 may also be changed from the direction shown in each figure.
• the processor 34 is illustrated, but in place of the processor 34 or together with the processor 34, at least one other CPU, at least one GPU, and/or at least one TPU is used. You can do it like this.
• the flight imaging program 90 and the composite image generation program 130 may be stored on a portable, non-transitory, computer-readable storage medium (hereinafter simply referred to as a "non-transitory storage medium") such as an SSD or a USB memory. may have been done.
• the flight imaging program 90 and/or the composite image generation program 130 stored on a non-transitory storage medium may be installed on the computer 26 of the flight imaging device 10.
• flight imaging program 90 and/or the composite image generation program 130 may be stored in a storage device such as another computer or a server device connected to the flight imaging device 10 via a network, and the flight imaging program 90 and/or the composite image generation program 130 may be stored in response to requests from the flight imaging device 10. Accordingly, flight imaging program 90 and/or composite image generation program 130 may be downloaded and installed on computer 26 .
• flight imaging program 90 and/or composite image generation program 130 it is not necessary to store all of the flight imaging program 90 and/or composite image generation program 130 in a storage device such as another computer or server device connected to the flight imaging device 10, or in the storage 36; Part of the program 90 and/or the composite image generation program 130 may be stored.
• the flight imaging device 10 has a built-in computer 26, the technology of the present disclosure is not limited to this, and for example, the computer 26 may be provided outside the flight imaging device 10.
• the computer 26 including the processor 34, the storage 36, and the RAM 38 is illustrated, but the technology of the present disclosure is not limited to this, and instead of the computer 26, an ASIC, an FPGA, and/or A device including a PLD may also be applied. Further, instead of the computer 26, a combination of hardware configuration and software configuration may be used.
• processors can be used as hardware resources for executing the various processes described in the above embodiments.
• the processor include a CPU, which is a general-purpose processor that functions as a hardware resource that executes various processes by executing software, that is, a program.
• the processor include a dedicated electronic circuit such as an FPGA, a PLD, or an ASIC, which is a processor having a circuit configuration specifically designed to execute a specific process.
• Each processor has a built-in memory or is connected to it, and each processor uses the memory to perform various processes.
• Hardware resources that execute various processes may be configured with one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a CPU and FPGA). Furthermore, the hardware resource that executes various processes may be one processor.
• one processor is configured by a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes various processes.
• a and/or B has the same meaning as “at least one of A and B.” That is, “A and/or B” means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, even when three or more items are expressed by connecting them with “and/or”, the same concept as “A and/or B" is applied.
• FIG. 1 (Additional note 1) Equipped with a processor, When a first imaging target area and a second imaging target area of a subject are imaged by the imaging device according to a moving position of a moving body on which an imaging device is mounted, the processor generating a composite image by combining a first captured image obtained by imaging the region and a second captured image obtained by capturing the second imaging target region; A first area that is a part of the first imaging target area overlaps a second area that is a part of the second imaging target area, The first imaged image is an image obtained by imaging the first imaged area by the imaging device while the first imaged area is irradiated with light by a light source mounted on the moving object.
• the intensity of the light irradiated onto the first region is higher than the intensity of the light irradiated around the first region
• the composite image includes pixel values of an image area corresponding to the second area of the second captured image as pixel values of an overlapping image area corresponding to the first area and the second area of the composite image.
• An image processing device that is an image using pixel values.
• the first imaged image is an image obtained by imaging the first imaged area by the imaging device while the first imaged area is irradiated with light by a light source mounted on the moving object.
• the intensity of the light irradiated onto the first region is higher than the intensity of the light irradiated around the first region
• the composite image includes pixel values of an image area corresponding to the second area of the second captured image as pixel values of an overlapping image area corresponding to the first area and the second area of the composite image.
• An image processing method that uses pixel values. (Additional note 3) A program that causes a computer to execute a process, The processing may be performed when a first imaging target area and a second imaging target area of a subject are imaged by the imaging device according to a moving position of a moving body on which an imaging device is mounted.
• the first imaged image is an image obtained by imaging the first imaged area by the imaging device while the first imaged area is irradiated with light by a light source mounted on the moving object. and The intensity of the light irradiated onto the first region is higher than the intensity of the light irradiated around the first region,
• the composite image includes pixel values of an image area corresponding to the second area of the second captured image as pixel values of an overlapping image area corresponding to the first area and the second area of the composite image.
• a program that is an image using pixel values an imaging device; a light source and a moving body equipped with the imaging device and the light source; Equipped with The imaging device images a first imaging target area and a second imaging target area of the subject according to a movement position of the moving object, A first area that is a part of the first imaging target area overlaps a second area that is a part of the second imaging target area, the light source irradiates the first imaging target area with the light; The intensity of the light irradiated onto the first region is higher than the intensity of the light irradiated around the first region.
• the mobile imaging device (Appendix 5) The mobile imaging device according to appendix 4, further comprising a control device that controls the imaging device and the light source.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Studio Devices (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=90099262

Family Applications (1)

Country Status (3)

Citations (2)

• 2023
  • 2023-05-09 JP JP2024543781A patent/JPWO2024047948A1/ja active Pending
  • 2023-05-09 WO PCT/JP2023/017455 patent/WO2024047948A1/ja not_active Ceased
• 2025
  • 2025-02-24 US US19/061,953 patent/US20250193530A1/en active Pending

Patent Citations (2)

Also Published As

Similar Documents

Legal Events