WO2023135910A1

WO2023135910A1 - Image-capturing device, image-capturing method, and program

Info

Publication number: WO2023135910A1
Application number: PCT/JP2022/041381
Authority: WO
Inventors: 健太郎宮本
Original assignee: 富士フイルム株式会社
Priority date: 2022-01-17
Filing date: 2022-11-07
Publication date: 2023-07-20

Abstract

According to the present invention, a processor acquires a first image, acquires a movement direction of a moving body on which an image-capturing device is mounted, acquires a second image obtained by performing image-capturing by the image-capturing device in a case in which the moving body has moved, determines an overlap ratio of the first image and the second image on the basis of a result of comparing a first feature point included in the first image and a second feature point included in the second image, and in a case of determining the overlap ratio, changes a first image region that is an image region included in the first image and is an image region for obtaining the first feature point and/or a second image region that is an image region included in the second image and is an image region for obtaining the second feature point, in accordance with the movement direction.

Description

IMAGING DEVICE, IMAGING METHOD, AND PROGRAM

The technology of the present disclosure relates to an imaging device, an imaging method, and a program.

JP 2010-045587 discloses an image capturing unit, an image display unit, a shake detection unit, an image recording unit, a relative relationship calculation unit, a display control unit, an overlap calculation unit, a notification unit, A camera device having a shooting control unit is disclosed. The image capturing unit captures an image. The image display unit has at least a screen that displays an image. The shake detection section detects apparatus shake during image shooting by the image shooting section. The image recording section records information of an image captured by the image capturing section. The relative relationship calculation unit calculates the photographing range of the first image photographed immediately before by the image photographing unit and recorded in the image recording unit, and the photographing range of the second image photographed by the image photographing unit subsequent to the first image. A relative degree parameter representing at least a relative positional relationship between is obtained. The display control unit generates an image for clearly indicating the relative positional relationship between the shooting ranges from the relative relationship degree parameter obtained by the relative relationship calculation unit, and displays the image together with the second image on the screen of the image display unit. display. The overlap calculation unit obtains a degree of overlap parameter representing the degree of overlap between the imaging range of the first image and the imaging range of the second image. The notification unit gives a predetermined notification to the photographer according to the redundancy parameter obtained by the overlap calculation unit. When the overlap parameter obtained by the overlap calculation unit is within a predetermined threshold range and it can be determined from the detection output of the shake detection unit that there is substantially no apparatus shake during image shooting by the image capturing unit, the image capturing unit to take an image.

Japanese Patent Application Laid-Open No. 2013-186853 describes an image processing device that includes acquisition means, division means, feature point detection means, vector calculation means, area selection means, and generation means. The acquisition means acquires a plurality of images continuously captured with movement of the imaging unit. The dividing means divides the image acquired by the acquiring means into a plurality of regions. The feature point detection means detects feature points in each of the plurality of areas divided by the division means. The vector calculation means calculates a plurality of vectors corresponding to a plurality of areas between adjacent images of the plurality of images based on the feature points detected by the feature point detection means. The area selection means selects a specific area from the plurality of divided areas based on the plurality of vectors calculated by the vector calculation means. The generation means generates a composite image by combining adjacent images based on the specific area selected by the area selection means.

International Publication No. 2018/168406 pamphlet describes an imaging control device for controlling imaging of a moving object equipped with a camera, which includes a wide-angle image acquisition unit, an imaging information acquisition unit, an overlap information acquisition unit, and an area information acquisition unit. A photographing control device is disclosed that includes a photographing area calculator, and a controller. The wide-angle image acquisition unit acquires a wide-angle image obtained by wide-angle imaging of an entire image of an imaging target. The photographing information acquisition unit acquires photographing information regarding the number of shots or the photographing angle of view of a plurality of divided images obtained by closely photographing a part of the whole image of the object to be photographed by a mobile camera. The margin information acquisition unit acquires margin information relating to margin when a composite image of a photographing target is generated by synthesizing a plurality of divided images. The area information acquisition unit acquires shooting target area information about an area of the entire image of the shooting target. Based on the shooting information, the margin information, and the shooting target region information, the shooting area calculation unit calculates each shot of a wide-angle image in which the margin is secured, which is the shooting area of each of the divided images constituting the composite image. Calculate the area. The control unit moves the moving body, causes the camera to take close-up shots of the calculated shooting areas, and acquires the shot close-up images as divided images. The control unit compares an image corresponding to each shooting area of the acquired wide-angle image with an image shot by the camera at close range, and controls the position of the moving body for causing the camera to take close shots of each shooting area.

Japanese Unexamined Patent Application Publication No. 2007-174301 discloses an image capturing device comprising a memory, motion detection means, guide image creation means, superimposed image creation means, and current image acquisition means. A memory stores the first captured image. The motion detection means detects motion of the imaging device and calculates a motion direction and a motion distance. The guide image creating means displays the first photographed image on the display screen, and shifts the displayed first photographed image in a direction opposite to the movement direction with respect to the display screen according to the movement distance. to fix the display, and the fixed image remaining on the display screen after the shift is displayed as a guide image. The superimposed image creating means superimposes and displays the current image and the guide image currently being captured by the image capturing device on the display screen. The current image obtaining means stores the current image in the memory as a second photographed image when it is detected that the photographing button has been pressed.

Japanese Patent Laying-Open No. 2008-252934 discloses a camera device having monitor means for monitoring an image that enters the field of view and means for electronically recording the captured image on a recording medium. The camera device comprises means for displaying on the monitor means an image showing the relative positional relationship between the image displayed on the monitor means and the image taken immediately before, together with the image within the field of view of the camera device. . Means for displaying an image showing the relative positional relationship on the monitor means together with an image within the field of view of the camera device, for the image displayed on the monitor means by a direction and amount corresponding to the relative positional relationship An image showing the moved image frame is displayed on the monitor means as an image showing the relative positional relationship.

One embodiment according to the technology of the present disclosure, for example, based on the result of comparing the first feature points included in the entire area of the first image and the second feature points included in the entire area of the second image, An imaging apparatus, an imaging method, and a program capable of reducing the load required for processor processing when determining an overlap rate compared to when determining an overlap rate between a first image and a second image offer.

A first aspect of the technology of the present disclosure is an imaging device that includes a processor, the processor acquires a first image, acquires a moving direction of a moving body on which the imaging device is mounted, and moves the moving body. acquire a second image captured by the imaging device, and compare the first feature points included in the first image with the second feature points included in the second image, based on the result of to determine the overlap ratio between the first image and the second image, and in the case of determining the overlap ratio, the image area included in the first image for obtaining the first feature point The imaging device changes a second image area, which is an image area included in a certain first image area and/or a second image and is an image area for obtaining a second feature point, according to a movement direction.

A second aspect of the technology of the present disclosure is the imaging device according to the first aspect, wherein the processor causes the display device to display the second image as a display image.

A third aspect of the technology of the present disclosure is the imaging device according to the first aspect or the second aspect, wherein the first image area is a partial image area of the first image, and the second image The region is an imaging device that is a partial image region of the second image.

A fourth aspect of the technology of the present disclosure is the imaging device according to any one of the first to third aspects, wherein the first image area is positioned on the front side in the moving direction of the first image. The second image area is an imaging device that is an area located on the rear side in the movement direction in the second image.

A fifth aspect of the technology of the present disclosure is the imaging device according to any one of the first to fourth aspects, wherein the processor divides the first image into a plurality of The imaging device selects a first image area from the first divided areas and selects a second image area from a plurality of second divided areas obtained by dividing the second image.

A sixth aspect of the technology of the present disclosure is the imaging device according to the fifth aspect, wherein the first image area includes two or more first divided areas among the plurality of first divided areas, and the second The image area is an imaging device that includes two or more of the plurality of second divided areas.

A seventh aspect of the technology of the present disclosure is the imaging device according to the fifth aspect, wherein the first image area is one first divided area of the plurality of first divided areas, and the second image area is an imaging device that is one second segmented area of a plurality of second segmented areas.

An eighth aspect of the technology of the present disclosure is the imaging device according to any one of the first to seventh aspects, wherein the first image area and the second image area are rectangular areas. It is an imaging device.

A ninth aspect of the technology of the present disclosure is the imaging device according to any one of the first to seventh aspects, wherein the processor moves in the horizontal direction or the vertical direction of the second image. is the direction of inclination, the first triangular region including the corner located on the front side in the moving direction among the four corners of the first image is set as the first image region, and the four corners of the second image are set to the first triangular region The imaging device sets the second triangular area including the corner positioned on the rear side in the movement direction as the second image area.

A tenth aspect of the technology of the present disclosure is the imaging device according to any one of the first to ninth aspects, wherein the processor acquires the first feature point and The imaging device obtains from the image sensor only image data used for comparison with the second feature point.

An eleventh aspect of the technology of the present disclosure is the imaging device according to any one of the first to tenth aspects, wherein the first feature point is a feature point included only in the first image area. , and the second feature point is an imaging device that is a feature point included only in the second image area.

A twelfth aspect of the technology of the present disclosure is the imaging device according to any one of the first to eleventh aspects, wherein the processor, in accordance with the moving speed of the moving body, An imaging device for changing the area and/or the area of the second image area.

A thirteenth aspect of the technology of the present disclosure is the imaging device according to the twelfth aspect, wherein the processor increases the area of the first image region and/or the area of the second image region as the moving speed of the moving object increases. is an imaging device that increases the

A fourteenth aspect of the technology of the present disclosure is the imaging device according to the twelfth aspect, wherein the processor increases the area of the first image region and/or the area of the second image region as the moving speed of the moving object increases. is an imaging device that reduces the

A fifteenth aspect of the technology of the present disclosure is the imaging device according to any one of the first to fourteenth aspects, wherein the processor, when the moving object is moving, is an imaging device that extracts a first feature point included in a first image area that is a part of the image area of the .

A sixteenth aspect of the technology of the present disclosure is the imaging device according to any one of the first to fifteenth aspects, wherein when the moving body is stationary, the processor It is an imaging device that extracts feature points included in an area.

A seventeenth aspect of the technology of the present disclosure is the imaging device according to any one of the first to sixteenth aspects, wherein the area of the first image region and the area of the second image region are adjacent to each other In the imaging device, each area is set to be larger than an overlap area, which is an area where the first images are overlapped when the first image is synthesized.

An eighteenth aspect of the technology of the present disclosure is an imaging device according to any one of the first to seventeenth aspects, wherein the moving object is a flying object.

A nineteenth aspect according to the technology of the present disclosure is to obtain a first image, to obtain a moving direction of a moving body on which an imaging device is mounted, and to capture an image by the imaging device when the moving body moves. obtaining a second image obtained by the above, and based on the result of comparing the first feature points included in the first image and the second feature points included in the second image, the first image and the second image and a first image area that is an image area included in the first image when determining the overlap rate and is an image area for obtaining the first feature point, and/or changing a second image area, which is an image area included in the second image and is an image area for obtaining the second feature points, according to the moving direction.

A twentieth aspect of the technology of the present disclosure is to obtain a first image, to obtain a moving direction of a moving body on which an imaging device is mounted, and to capture an image by the imaging device when the moving body moves. obtaining a second image obtained by the above, and based on the result of comparing the first feature points included in the first image and the second feature points included in the second image, the first image and the second image and a first image area that is an image area included in the first image when determining the overlap rate and is an image area for obtaining the first feature point, and/or for causing the computer to perform processing including changing the second image area, which is an image area included in the second image and is an image area for obtaining the second feature points, according to the movement direction. It's a program.

It is a perspective view which shows an example of a flight imaging device. It is a block diagram which shows an example of the hardware constitutions of an imaging device. 2 is a block diagram showing an example of a functional configuration of an imaging device; FIG. FIG. 4 is an explanatory diagram illustrating an example of a first operation of a processor; It is an explanatory view explaining an example of the 2nd operation of a processor. It is an explanatory view explaining an example of the 3rd operation of a processor. It is an explanatory view explaining an example of the 4th operation of a processor. FIG. 20 is an explanatory diagram illustrating an example of a fifth operation of the processor; FIG. 20 is an explanatory diagram illustrating an example of a sixth operation of the processor; FIG. 20 is an explanatory diagram illustrating an example of a seventh operation of the processor; FIG. 22 is an explanatory diagram illustrating an example of an eighth operation of the processor; FIG. 22 is an explanatory diagram illustrating an example of a ninth operation of the processor; FIG. 22 is an explanatory diagram illustrating an example of a tenth operation of the processor; FIG. 22 is an explanatory diagram illustrating an example of an eleventh operation of the processor; It is a perspective view which shows an example of the moving direction of a flight imaging device. FIG. 4 is an explanatory diagram illustrating a first example of a relationship between an image region in which feature points are compared between an image for synthesis and an image for display, and a moving direction of the flight imaging device; FIG. 10 is an explanatory diagram for explaining a second example of the relationship between image regions in which feature points are compared between an image for synthesis and an image for display and the moving direction of the flight imaging device; 4 is a flowchart showing an example of the flow of imaging processing; FIG. 11 is an explanatory diagram for explaining a first modification of the operation of the processor; FIG. 11 is an explanatory diagram for explaining a second modification of the operation of the processor; FIG. 11 is an explanatory diagram for explaining a third modification of the operation of the processor; FIG. 14 is an explanatory diagram for explaining a fourth modification of the operation of the processor; FIG. 14 is an explanatory diagram for explaining a fifth modification of the operation of the processor;

An example of an embodiment of an imaging device, an imaging method, and a program according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

First, the wording used in the following explanation will be explained.

　I/F is an abbreviation for "Interface". RAM is an abbreviation for "Random Access Memory". EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". CPU is an abbreviation for "Central Processing Unit". HDD is an abbreviation for "Hard Disk Drive". SSD is an abbreviation for "Solid State Drive". DRAM is an abbreviation for "Dynamic Random Access Memory". SRAM is an abbreviation for "Static Random Access Memory". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". LiDAR is an abbreviation for “Light Detection And Ranging”. GPU is an abbreviation for "Graphics Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". USB is an abbreviation for "Universal Serial Bus". ASIC is an abbreviation for "Application Specific Integrated Circuit". FPGA is an abbreviation for "Field-Programmable Gate Array". PLD is an abbreviation for "Programmable Logic Device". SoC is an abbreviation for "System-on-a-chip." IC is an abbreviation for "Integrated Circuit".

In the description of this specification, "perpendicular" means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to perfect verticality, and does not go against the spirit of the technology of the present disclosure. It refers to the vertical in the sense of including the error of In the description of this specification, "match" means, in addition to perfect match, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not go against the spirit of the technology of the present disclosure. It refers to a match in terms of meaning including errors in In the description of this specification, "equivalent" means, in addition to complete equivalent, an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, and does not violate the spirit of the technology of the present disclosure It refers to equality in the sense of including the error of In the description of this specification, the “horizontal direction” means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs in addition to the complete horizontal direction, and is contrary to the spirit of the technology of the present disclosure. It refers to the horizontal direction in the sense of including the degree of error that does not occur. In the description of this specification, the “vertical direction” means an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the perfect vertical direction, and is contrary to the spirit of the technology of the present disclosure. It refers to the vertical direction in the sense of including the degree of error that does not occur.

As shown in FIG. 1 as an example, the flight imaging device 1 has a flight function and an imaging function, and images the wall surface 2A of the object 2 while flying. In the description of this specification, the concept of “flight” includes not only the meaning that the flight imaging device 1 moves in the air, but also the meaning that the flight imaging device 1 stops in the air.

The wall surface 2A is flat as an example. A plane refers to a two-dimensional surface (that is, a surface along a two-dimensional direction). Moreover, in the description of this specification, the concept of "flat surface" does not include the meaning of a mirror surface. In the present embodiment, for example, the wall surface 2A is a plane defined horizontally and vertically (that is, a plane extending horizontally and vertically). The wall surface 2A includes irregularities. The unevenness referred to here includes, for example, unevenness due to defects and/or defects in addition to unevenness caused by the material forming the wall surface 2A. As an example, the object 2 having the wall surface 2A is a pier provided on a bridge. The piers are made of reinforced concrete, for example. Although a bridge pier is given as an example of the object 2 here, the object 2 may be an object other than a bridge pier (for example, a tunnel or a dam).

The flight function of the flight imaging device 1 (hereinafter also simply referred to as "flight function") is a function of the flight imaging device 1 flying based on a flight instruction signal. A flight instruction signal refers to a signal that instructs the flight imaging device 1 to fly. The flight instruction signal is transmitted, for example, from a transmitter 20 for operating the flight imaging device 1 . The transmitter 20 is operated by a user (not shown). The transmitter 20 includes a control unit 22 for operating the flight imaging device 1 and a display device 24 for displaying images captured by the flight imaging device 1 . The display device 24 is, for example, a liquid crystal display.

Specifically, the flight instruction signal is classified into a plurality of instruction signals including a movement instruction signal that instructs the movement and movement direction of the flight imaging device 1 and a stationary instruction signal that instructs the flight imaging device 1 to stand still. . Here, an example in which the flight instruction signal is transmitted from the transmitter 20 is given. good. The imaging function of the flight imaging device 1 (hereinafter also simply referred to as “imaging function”) is a function of the flight imaging device 1 imaging a subject (as an example, the wall surface 2A of the target object 2).

The flight imaging device 1 includes a flying object 10 and an imaging device 30. The flying object 10 is, for example, an unmanned aerial vehicle such as a drone. A flight function is realized by the aircraft 10 . The flying object 10 has a plurality of propellers 12 and flies by rotating the plurality of propellers 12 . Flying of the aircraft 10 is synonymous with flying of the flight imaging device 1 . The flying object 10 is an example of the “moving object” and the “flying object” according to the technology of the present disclosure.

The imaging device 30 is, for example, a digital camera or a video camera. An imaging function is realized by the imaging device 30 . The imaging device 30 is an example of an “imaging device” according to the technology of the present disclosure. The imaging device 30 is mounted on the aircraft 10 . Specifically, the imaging device 30 is provided below the aircraft 10 . Here, an example in which the imaging device 30 is provided in the lower part of the flying object 10 is given, but the imaging device 30 may be provided in the upper part or the front part of the flying object 10 or the like.

The flight imaging device 1 sequentially images a plurality of areas 3 of the wall surface 2A. A region 3 is a region determined by the angle of view of the flight imaging device 1 . In the example shown in FIG. 1, a rectangular area is shown as an example of the area 3 . A plurality of synthesis images 92 are obtained by sequentially capturing images of the plurality of regions 3 by the imaging device 30 . A composite image 90 is generated by combining a plurality of composite images 92 . A plurality of images for synthesis 92 are synthesized so that adjacent images for synthesis 92 partially overlap each other. The process of synthesizing a plurality of images for synthesis 92 may be performed by the flight imaging device 1 or may be performed by an external device (not shown) communicably connected to the flight imaging device 1 . The composite image 90 is used, for example, for inspecting or surveying the wall surface 2A of the object 2 .

The plurality of synthesized images 90 used to generate the synthesized image 90 also include images that have undergone projective transformation. An image that has undergone projective transformation refers to, for example, an image that includes an image area distorted into a trapezoid or the like due to the posture (for example, depression angle or elevation angle) of the imaging device 30, and is corrected. In projective transformation, the wall surface 2A is imaged by the imaging device 30 in a state in which the posture of the imaging device 30 is tilted with respect to the wall surface 2A (that is, a state in which the optical axis OA of the imaging device 30 is tilted with respect to the wall surface 2A). This is the processing performed on the image obtained by

The distortion of the image caused by the angle of depression or elevation is corrected by performing a projective transformation. That is, the image obtained by the imaging device 30 imaging with the posture of the imaging device 30 tilted with respect to the wall surface 2A is transformed into an image as if from a position facing the wall surface 2A. It is converted like an image obtained by taking an image.

In the example shown in FIG. 1, each area 3 is imaged by the imaging device 30 with the optical axis OA of the imaging device 30 perpendicular to the wall surface 2A. The plurality of regions 3 are imaged such that adjacent regions 3 partially overlap each other. A plurality of regions 3 are imaged so that the adjacent regions 3 partially overlap each other. This is for synthesizing

Hereafter, overlapping of adjacent regions 3 may be referred to as overlap. Also, the ratio of the area of the overlapping portion to the total area of each region 3 is called an overlap ratio. The overlap ratio is set to the default overlap ratio. The default overlap rate is set to a rate (eg, 30%) at which the amount of feature points that can be combined with adjacent composite images 92 is obtained.

In the example shown in FIG. 1 , the plurality of regions 3 are regions 3 that have already been imaged (that is, regions 3 that have been imaged by the flight imaging device 1) and regions 3 that have not been imaged (that is, regions 3 that have been imaged by the flight imaging device 1). 3). Hereinafter, when the plurality of regions 3 are separately described, the region 3 that has not yet been imaged among the plurality of regions 3 will be referred to as "imaging target region 3A", and the region 3 that has already been imaged among the plurality of regions 3 will be referred to as " This is referred to as an imaged area 3B'.

As an example, the flight imaging device 1 images a plurality of regions 3 while moving in a zigzag manner by alternately repeating horizontal movement and vertical movement. In addition, the flight imaging device 1 captures a plurality of images in the order in which a part of the imaging target area 3A and a part of the imaging completed area 3B that is imaged one frame before the imaging target area 3A (for example, one frame before) overlap each other. Each area 3 is imaged. In the following, as an example, as shown in FIG. 1, the flight imaging device 1 alternately repeats movement in the horizontal direction and the movement in the vertical direction, thereby capturing images of a plurality of regions 3 while moving in a zigzag manner. do.

As shown in FIG. 2 as an example, the imaging device 30 includes a computer 32, a communication device 34, an image sensor 36, an image sensor driver 38, an imaging lens 40, an image memory 42, and an input/output I/F44.

The computer 32 includes a processor 46, storage 48, and RAM50. Processor 46 , storage 48 , and RAM 50 are interconnected via bus 52 , and bus 52 is connected to input/output I/F 44 . Also, the input/output I/F 44 is connected with the communication device 34 , the image sensor driver 38 , the imaging lens 40 , and the image memory 42 . The computer 32 is an example of a "computer" according to the technology of the present disclosure. The processor 46 is an example of a "processor" according to the technology of the present disclosure.

The processor 46 has, for example, a CPU and controls the imaging device 30 as a whole. The storage 48 is a nonvolatile storage device that stores various programs, various parameters, and the like. The storage 48 includes, for example, HDD and/or flash memory (eg, EEPROM and/or SSD).

The RAM 50 is a memory that temporarily stores information, and is used by the processor 46 as a work memory. Examples of the RAM 50 include DRAM and/or SRAM.

The communication device 34 is communicably connected to the transmitter 20 as an example. Here, the communication device 34 is wirelessly communicably connected to the transmitter 20 according to a predetermined wireless communication standard. The predefined wireless communication standard includes, for example, Wi-Fi (registered trademark) or Bluetooth (registered trademark). The communication device 34 is in charge of exchanging information with the transmitter 20 . For example, communication device 34 transmits information to transmitter 20 in response to a request from processor 46 . Communication device 34 also receives information transmitted from transmitter 20 and outputs the received information to processor 46 via bus 52 . Although an example in which the communication device 34 is communicably connected to the transmitter 20 is given here, the communication device 34 may be communicably connected to the transmitter 20 and/or the aircraft 10. .

The image sensor 36 is connected with an image sensor driver 38 . Image sensor driver 38 controls image sensor 36 according to instructions from processor 46 . Image sensor 36 is, for example, a CMOS image sensor. Although a CMOS image sensor is exemplified as the image sensor 36 here, the technology of the present disclosure is not limited to this, and other image sensors may be used. The image sensor 36 captures an image of a subject (for example, the wall surface 2A of the target object 2) under the control of an image sensor driver 38, and outputs image data obtained by capturing the image.

The imaging lens 40 is arranged on the subject side (object side) of the image sensor 36 . The imaging lens 40 captures subject light, which is reflected light from the subject, and forms an image of the captured subject light on the imaging surface of the image sensor 36 . The imaging lens 40 includes a plurality of optical elements (not shown) such as a focus lens, a zoom lens, and an aperture. The imaging lens 40 is connected to the computer 32 via an input/output I/F 44 . Specifically, the plurality of optical elements included in the imaging lens 40 are connected to the input/output I/F 44 via a driving mechanism (not shown) having a power source. A plurality of optical elements included in imaging lens 40 operate under the control of computer 32 . In the imaging device 30, by operating a plurality of optical elements included in the imaging lens 40, adjustment of focus, optical zoom, shutter speed, and the like are realized.

The image memory 42 temporarily stores image data generated by the image sensor 36 . The processor 46 acquires image data from the image memory 42 and uses the acquired image data to perform various processes.

As shown in FIG. 3 as an example, the storage 48 stores an imaging program 60 . The imaging program 60 is an example of a "program" according to the technology of the present disclosure. The processor 46 reads the imaging program 60 from the storage 48 and executes the read imaging program 60 on the RAM 50 . The processor 46 performs imaging processing for imaging a plurality of areas 3 (see FIG. 1) according to an imaging program 60 executed on the RAM 50 .

The imaging process is performed by the processor 46 according to the imaging program 60, including a first imaging control unit 62, a first feature point information generation unit 64, a first movement determination unit 66, a movement direction information generation unit 68, a second imaging control unit 70, a display It is realized by operating as the control unit 72 , the second feature point information generation unit 74 , the feature point comparison unit 76 , the overlap determination unit 78 , and the second movement determination unit 80 .

As an example, as shown in FIG. 4, the flying object 10 receives the movement instruction signal transmitted from the transmitter 20 in response to the user's operation, and moves to the first imaging position based on the received movement instruction signal. Also, the flying object 10 receives a stationary instruction signal transmitted from the transmitter 20 in response to an operation by the user, and stands still at the initial imaging position based on the received stationary instruction signal. When the imaging device 30 receives the imaging start signal transmitted from the transmitter 20 in response to the user's operation, the imaging device 30 executes imaging processing described below.

The first imaging control unit 62 outputs a first imaging instruction signal to the image sensor 36 to cause the image sensor 36 to image the imaging target region 3A. Image data for synthesis is obtained by imaging the imaging target region 3A with the image sensor 36 under the control of the first imaging control section 62 . The synthesizing image data is image data representing the synthesizing image 92 . The image data for synthesis is stored in the storage 48 . The image for synthesis 92 indicated by the image data for synthesis shown in FIG. 4 is the first image for synthesis. The composition image 92 is an example of the "first image" according to the technology of the present disclosure. The synthesizing image 92 includes characteristic points corresponding to the unevenness of the imaging target area 3A. Hereinafter, the feature point included in the image for synthesis 92 will be referred to as a "first feature point".

When the image for synthesis 92 is the first image for synthesis, the first feature point information generation unit 64 acquires the image for synthesis 92 based on the image data for synthesis stored in the storage 48, and The image for use 92 is divided into a plurality of first divided areas 96 . As an example, the composite image 92 is evenly divided into a plurality of first divided regions 96 . In the example shown in FIG. 4, the number of multiple first divided areas 96 is nine. In the following, when the nine first segmented regions 96 need to be distinguished and explained, the nine first segmented regions 96 are respectively divided into the first segmented region 96-1 and the second segmented region 96-1 by using branch numbers 1 to 9. 1 divided area 96-2, first divided area 96-3, first divided area 96-4, first divided area 96-5, first divided area 96-6, first divided area 96-7, first divided area They are referred to as a region 96-8 and a first segmented region 96-9. As an example, each of the plurality of first divided regions 96 is a rectangular region.

When the image for synthesis 92 is the first image for synthesis, the first feature point information generation unit 64 extracts the first feature points included in the entire area of the image for synthesis 92 for each first divided area 96. , first feature point information indicating the coordinates of the first feature point for each of the extracted first divided areas 96 is generated. The first feature point information generated by the first feature point information generation unit 64 is stored in the storage 48 . The coordinates of the first feature point indicated by the first feature point information are derived, for example, by performing image processing (for example, high-frequency component extraction processing, etc.) on the synthesis image data. The coordinates of the first feature point are, for example, coordinates based on any one of the four vertices of the imaging target area 3A.

As an example, as shown in FIG. 5, when the aircraft 10 receives the movement instruction signal transmitted from the transmitter 20 in response to the user's operation, the aircraft 10 moves based on the received movement instruction signal. In the example shown in FIG. 5, the flying object 10 is moving horizontally based on the movement instruction signal. Specifically, the moving direction of the flying object 10 is the right direction.

The first movement determination unit 66 determines whether the flying object 10 is moving. The first movement determination unit 66 determines that the flying object 10 is moving based on the movement instruction signal when the imaging device 30 receives the movement instruction signal transmitted from the transmitter 20 in response to the user's operation. do. The movement instruction signal includes information for instructing the movement direction of the aircraft 10 .

When the first movement determining unit 66 determines that the flying object 10 is moving, the moving direction information generating unit 68 acquires the moving direction of the flying object 10 based on the movement instruction signal, and converts the acquired moving direction to Generate moving direction information to indicate.

As an example, as shown in FIG. 6, the flying object 10 continues to move based on the received movement instruction signal while receiving the movement instruction signal transmitted from the transmitter 20 in response to the user's operation.

The second imaging control unit 70 outputs a second imaging instruction signal to the image sensor 36 to cause the image sensor 36 to image the imaging target region 3A. Display image data is obtained by capturing an image of the image capturing target area 3A with the image sensor 36 under the control of the second image capturing control section 70 . The display image data is image data representing the display image 94 . The display image 94 is obtained by being imaged by the imaging device 30 when the flying object 10 moves from the position where the synthesis image 92 was obtained. The image data for display is stored in the image memory 42 . The display image 94 is an example of the "second image" and the "display image" according to the technology of the present disclosure. The display image 94 includes characteristic points corresponding to the unevenness of the imaging target area 3A. Hereinafter, the feature points included in the display image 94 are referred to as "second feature points". Further, hereinafter, when it is not necessary to distinguish between the "first feature point" and the "second feature point", they are also simply referred to as the "feature point".

The display control unit 72 acquires display image data stored in the image memory 42 and executes various processes on the display image data. The display control unit 72 then outputs the display image data to the transmitter 20 . The transmitter 20 receives the display image data, and displays the display image 94 (that is, the live view image) on the display device 24 based on the received display image data.

The second feature point information generation unit 74 acquires the display image 94 based on the display image data stored in the image memory 42 and divides the acquired display image 94 into a plurality of second divided areas 100 . The display image 94 is equally divided into a plurality of second divided regions 100 as an example. The number of the plurality of second divided regions 100 is the same as the number of the plurality of first divided regions 96 . That is, in the example shown in FIG. 6, the number of the plurality of second divided regions 100 is nine. In the following, when the nine second divided regions 100 need to be distinguished and explained, the nine second divided regions 100 are divided into the second divided region 100-1 and the second divided region 100-1 by using branch numbers 1 to 9, respectively. Two divided regions 100-2, second divided regions 100-3, second divided regions 100-4, second divided regions 100-5, second divided regions 100-6, second divided regions 100-7, second divided regions They are called an area 100-8 and a second divided area 100-9. As an example, each of the plurality of second divided regions 100 is a rectangular region.

The second feature point information generation unit 74 generates the second split areas 100 for extracting the second feature points from the plurality of second split areas 100 obtained by splitting the display image 94 as second image areas. 102. Specifically, the second feature point information generation unit 74 acquires the movement direction of the flying object 10 (hereinafter also simply referred to as “movement direction”) based on the movement direction information generated by the movement direction information generation unit 68. Then, the second segmented region 100 located on the rear side in the movement direction from among the plurality of second segmented regions 100 is selected as the second image region 102 . In the example shown in FIG. 6, three second divided regions 100 (as an example, a second divided region 100-1, a second divided region 100- 4, and the second segmented area 100-7) is selected as the second image area 102. FIG.

Subsequently, the second feature point information generation unit 74 extracts second feature points included in the second image area 102 for each second image area 102 . A second feature point is a feature point included only in the second image area 102 . Then, the second feature point information generation unit 74 generates second feature point information indicating the coordinates of the second feature points for each of the extracted second image regions 102 . The second feature point information generated by the second feature point information generating section 74 is stored in the storage 48 . The coordinates of the second feature points extracted by the second feature point information generating section 74 are derived by the same method as the coordinates of the first feature points extracted by the first feature point information generating section 64 .

As an example, FIG. 7 shows the operation of the feature point comparison unit 76 when the first composite image is obtained as the composite image 92 . When the first image for synthesis is obtained as the image for synthesis 92, the first feature point information stored in the storage 48 corresponding to the first image for synthesis is the entire area of the image for synthesis 92. is information indicating the coordinates of the first feature point for each of the first divided areas 96 .

The feature point comparison unit 76 acquires the first feature point information and the second feature point information stored in the storage 48 . Further, when the first composite image is obtained as the composite image 92 , the feature point comparison section 76 acquires the movement direction information generated by the movement direction information generation section 68 . Then, based on the acquired moving direction information, the feature point comparison unit 76 selects the first divided area 96 positioned forward in the moving direction from among the plurality of first divided areas 96 with respect to the synthesis image 92 as the first divided area 96 . One image area 98 is selected. In the example shown in FIG. 7, three first divided regions 96 (as an example, the first divided region 96-3, the first divided region 96-6 , and the first segmented area 96 - 9 ) are selected as the first image area 98 .

Subsequently, the feature point comparison unit 76 extracts the first feature points included in the first image area 98 from the first feature point information. The first feature point is a feature point included only in the first image area 98 . Then, the feature point comparison unit 76 compares the first feature points included in the first image area 98 and the second feature points included in the second image area 102 . That is, the feature point comparison unit 76 compares the first feature point included in the first image area 98 of the image for synthesis 92 and the first feature point of the image for display 94 among the image for synthesis 92 and the image for display 94 captured at different times. The second feature points included in the second image area 102 are compared. The feature point comparison unit 76 generates comparison result information indicating a result of comparing the first feature point and the second feature point. The comparison result information is specifically information about the result of comparing the coordinates of the first feature point and the coordinates of the second feature point. In addition, in order to synthesize adjacent images 92 for synthesis among the plurality of synthesized images 90 based on the first feature points and the second feature points, the area of the three first image regions 98 and the area of the three second image regions The area of 102 is set to be larger than the overlap area, which is the area where the adjacent images for synthesis 92 are overlapped when they are synthesized.

As an example, as shown in FIG. 8, the overlap determination unit 78 acquires the comparison result information generated by the feature point comparison unit 76, and based on the acquired comparison result information, an image 92 for synthesis and an image 94 for display. It is determined whether or not the overlap rate of the overlap region 95, which is the region where the and overlap, is equal to or less than a predetermined threshold. Here, the overlap ratio refers to, for example, the ratio of the area of the overlap region 95 to the area of one frame (for example, the area of the image for synthesis 92 or the area of the image for display 94). An overlap region 95 between the image for synthesis 92 and the image for display 94 corresponds to the overlap region 5 where the regions 3 overlap each other. The predetermined threshold value is set in consideration of the efficiency of sequentially imaging a plurality of regions 3 and the aforementioned predetermined overlap ratio for synthesizing adjacent images for synthesis 92 (see FIG. 1). . For example, the predetermined threshold is set to 50% or less in consideration of the efficiency when sequentially imaging a plurality of regions 3 . Also, the default threshold is set to a value greater than the default overlap (30% as an example). Note that the moving speed of the flying object 10 is the speed at which the overlap determination unit 78 performs at least one determination until the overlap ratio between the image for synthesis 92 and the image for display 94 falls below the predetermined overlap ratio. is set to

As an example, as shown in FIG. 9, when the overlap determination unit 78 determines that the overlap ratio between the image for synthesis 92 and the image for display 94 is equal to or less than a predetermined threshold, the first imaging control unit 62 By outputting the first imaging instruction signal to the image sensor 36, the image sensor 36 is caused to image the imaging target region 3A. Image data for synthesis is obtained by imaging the imaging target region 3A with the image sensor 36 under the control of the first imaging control section 62 . A synthesizing image 92 indicated by the synthesizing image data shown in FIG. 9 is the synthesizing image 92 obtained after the above-described first synthesizing image (see FIG. 4). An image for synthesis obtained after the first image for synthesis is hereinafter referred to as the Nth image for synthesis (where N≧2). The image data for synthesis representing the Nth image for synthesis is stored in the storage 48 .

Note that, when the overlap determination unit 78 determines that the overlap ratio between the image for synthesis 92 and the image for display 94 is equal to or less than a predetermined threshold value, the first imaging control unit 62 changes the image from the imaging device 30 to the transmitter 20 . may be caused to transmit an imaging permission signal. Further, the transmitter 20 may display, on the display device 24, characters or the like indicating that the imaging is permitted when the imaging permission signal is received. Then, when the imaging device 30 receives an imaging instruction signal transmitted from the transmitter 20 to the imaging device 30 by the user according to characters or the like displayed on the display device 24, the first imaging control unit 62 By outputting the first imaging instruction signal to the sensor 36, the image sensor 36 may be caused to image the imaging target region 3A.

As an example, as shown in FIG. 10, the aircraft 10 continues to move based on the received movement instruction signal while receiving the movement instruction signal as the flight instruction signal.

The second movement determination unit 80 determines whether the flying object 10 continues to move based on the movement instruction signal received by the imaging device 30 and the movement direction information generated by the movement direction information generation unit 68. judge. When the movement instruction signal is received by the imaging device 30 and the movement direction indicated by the movement instruction signal matches the movement direction indicated by the movement direction information, the second movement determination unit 80 determines whether the flying object 10 continues to move. determine that there is

When the second movement determination unit 80 determines that the flying object 10 continues to move, the first feature point information generation unit 64 generates a synthesis image data based on the latest synthesis image data stored in the storage 48 . An image 92 (that is, the N-th image for synthesis) is obtained, and the obtained image for synthesis 92 is divided into a plurality of first divided regions 96 . As an example, the composite image 92 is evenly divided into a plurality of first divided regions 96 . As an example, the number of the plurality of first divided regions 96 is the same as the number of divisions of the first composite image. That is, in the example shown in FIG. 10, the number of multiple first divided areas 96 is nine. As an example, each of the plurality of first divided regions 96 is a rectangular region.

When the second movement determination unit 80 determines that the flying object 10 continues to move, the first feature point information generation unit 64 generates a plurality of first feature point information obtained by dividing the synthesis image 92 . A first divided area 96 for extracting a first feature point from the divided area 96 is selected as a first image area 98 . Specifically, the first feature point information generation unit 64 acquires the movement direction of the flying object 10 based on the movement direction information generated by the movement direction information generation unit 68, and A first divided area 96 located on the front side in the moving direction of is selected as a first image area 98 . In the example shown in FIG. 10, three first divided regions 96 (as an example, the first divided region 96-3, the first divided region 96-6 , and the first segmented area 96 - 9 ) are selected as the first image area 98 .

Subsequently, the first feature point information generation unit 64 extracts first feature points included in the first image area 98 for each first image area 98 . The first feature point is a feature point included only in the first image area 98 . In this manner, the first feature point information generation unit 64 generates the first feature points included in the first image area 98, which is a partial image area of the composite image 90, when the flying object 10 is moving. Each first image area 98 is extracted. Then, the first feature point information generating section 64 generates first feature point information indicating the coordinates of the first feature points for each of the extracted first image regions 98 . The first feature point information generated by the first feature point information generation unit 64 is stored in the storage 48 .

As an example, FIG. 11 shows the operation of the feature point comparison unit 76 when the N-th synthesis image is obtained as the synthesis image 92 while the flying object 10 continues to move (see FIG. 9). ing. When the flying object 10 continues to move, the first feature point information corresponding to the N-th synthesis image is the first segmented region located on the front side of the plurality of first segmented regions 96 in the moving direction. 96 is information indicating the coordinates of the first feature point for each first image area 98 . On the other hand, the second feature point information is information indicating the coordinates of the second feature point for each second image region 102 which is the second segmented region 100 located on the rear side in the moving direction among the plurality of second segmented regions 100. is.

The feature point comparison unit 76 acquires the first feature point information and the second feature point information stored in the storage 48 . Then, when the flying object 10 is moving, the feature point comparison unit 76 compares the first feature points and the second image included in the first image area 98 based on the first feature point information and the second feature point information. A second feature point included in the area 102 is compared, and comparison result information indicating the result of the comparison is generated.

As an example, as shown in FIG. 12, even when the flying object 10 reaches a position where the imaging device 30 captures an image of the region 3 located at the end of the plurality of regions 3, the first imaging control unit 62 When the overlap determination unit 78 determines that the overlap ratio between the image for synthesis 92 and the image for display 94 is equal to or less than the predetermined threshold value, by outputting the first imaging instruction signal to the image sensor 36, The image sensor 36 is made to image the imaging target area 3A. Image data for synthesis is obtained by imaging the imaging target region 3A with the image sensor 36 under the control of the first imaging control section 62 . A composite image 92 indicated by the composite image data shown in FIG. 11 is the N-th composite image. The image data for synthesis representing the Nth image for synthesis is stored in the storage 48 .

As an example, as shown in FIG. 13 , when the flying object 10 reaches a position where the image capturing device 30 captures an image of the area 3 located at the end of the plurality of areas 3, the transmitter is operated in response to the user's operation. A stationary instruction signal is transmitted from 20 to the aircraft 10 as a flight instruction signal.

When the imaging device 30 receives a stationary instruction signal as the flight instruction signal, the second movement determination unit 80 determines that the flying object 10 has not continued to move, that is, the flying object 10 has stopped moving.

When the second movement determination unit 80 determines that the flying object 10 has stopped moving, the first feature point information generation unit 64 generates the image for synthesis 92 based on the latest image data for synthesis stored in the storage 48 . (that is, the N-th image for synthesis) is obtained, and the obtained image for synthesis 92 is divided into a plurality of first divided regions 96 . As an example, the composite image 92 is evenly divided into a plurality of first divided regions 96 . As an example, the number of the plurality of first divided regions 96 is the same as the number of divisions of the first composite image. That is, in the example shown in FIG. 12, the number of multiple first divided areas 96 is nine. As an example, each of the plurality of first divided regions 96 is a rectangular region.

Then, when the second movement determination unit 80 determines that the flying object 10 has stopped moving, the first feature point information generation unit 64 converts the first feature points included in the entire area of the synthesis image 92 into the first Each divided area 96 is extracted, and first characteristic point information indicating the coordinates of the first characteristic point for each first divided area 96 extracted is generated. In this way, the first feature point information generator 64 extracts the first feature points included in the entire area of the synthesis image 92 for each first divided area 96 when the flying object 10 stops moving. Then, the first feature point information generating section 64 generates first feature point information indicating the coordinates of the first feature point for each of the extracted first divided areas 96 . The first feature point information generated by the first feature point information generation unit 64 is stored in the storage 48 .

As an example, FIG. 14 shows the feature point comparison unit 76 when the flying object 10 temporarily stops moving after obtaining the N-th image for synthesis (see FIG. 12) as the image for synthesis 92 and then starts moving. operation is shown. In the example shown in FIG. 14, the flying object 10 is moving vertically based on the movement instruction signal. Specifically, the moving direction of the flying object 10 is downward. When the flying object 10 stops moving, the first feature point information corresponding to the Nth image for synthesis indicates the coordinates of the first feature point for each first divided area 96 for the entire area of the image for synthesis 92. Information. On the other hand, the second feature point information is information indicating the coordinates of the second feature point for each second image region 102 which is the second segmented region 100 located on the rear side in the moving direction among the plurality of second segmented regions 100. is.

The feature point comparison unit 76 acquires the first feature point information and the second feature point information stored in the storage 48 . Further, when the flying object 10 stops moving, the feature point comparison unit 76 acquires the movement direction information generated by the movement direction information generation unit 68, and based on the acquired movement direction information, the image for synthesis 92 is generated. On the other hand, the first divided area 96 located on the front side in the moving direction from among the plurality of first divided areas 96 is selected as the first image area 98 . In the example shown in FIG. 14, three first segmented regions 96 (for example, first segmented region 96-7, first segmented region 96-8 , and the first segmented area 96 - 9 ) are selected as the first image area 98 .

Subsequently, the feature point comparison unit 76 extracts the first feature points included in the first image area 98 from the first feature point information. Then, the feature point comparison unit 76 compares the first feature points included in the first image area 98 and the second feature points included in the second image area 102, and generates comparison result information indicating the comparison result. do.

Note that the above description assumes an example (see FIG. 1) of imaging a plurality of regions 3 while moving in a zigzag manner by alternately repeating movement in the horizontal direction and movement in the vertical direction. there is However, the flight imaging device 1 may image the wall surface 2A while moving in all directions along the wall surface 2A of the object 2 .

As an example, as shown in FIG. 15, the movement directions of the flight imaging device 1 are classified into eight directions along the wall surface 2A of the object 2. In the example shown in FIG. 15 , rightward, leftward, upward, downward, upper right, lower right, upper left, and lower left directions are shown as an example of the moving direction of the flight imaging device 1 . That is, the flight imaging device 1 moves along the designated direction among the right direction, left direction, upward direction, downward direction, upper right direction, lower right direction, upper left direction, and lower left direction.

As an example, in FIG. 16 , when the movement direction of the flight imaging device 1 is the right direction, the left direction, the upward direction, and the downward direction, the characteristic point comparison unit 76 determines the characteristic points according to the movement direction of the flying object 10 . A relationship is shown between a first image area 98 and a second image area 102 for which .

For example, when the moving direction of the flying object 10 is the right direction, the feature point comparison unit 76 sets the first segmented region 96-3, the first segmented region 96-6, and the first segmented region 96-3 as the first image region 98. 96-9, and the second feature points included in the second divided regions 100-1, 100-4, and 100-7 as the second image region 102. Compare with

Further, when the moving direction of the flying object 10 is the left direction, the feature point comparison unit 76 selects the first segmented region 96-1, the first segmented region 96-4, and the first segmented region as the first image region 98. 96-7, and the second feature points included in the second divided regions 100-3, 100-6, and 100-9 as the second image region 102 Compare with

Further, when the moving direction of the flying object 10 is upward, the feature point comparison unit 76 selects the first segmented region 96-1, the first segmented region 96-2, and the first segmented region as the first image region 98. 96-3, and the second feature points included in the second divided regions 100-7, 100-8, and 100-9 as the second image region 102 Compare with

Further, when the moving direction of the flying object 10 is downward, the feature point comparison unit 76 selects the first segmented region 96-7, the first segmented region 96-8, and the first segmented region 96-7 as the first image region 98. 96-9, and the second feature points included in the second divided regions 100-1, 100-2, and 100-3 as the second image region 102 Compare with

As an example, in FIG. 17 , when the movement direction of the flight imaging device 1 is the upper right direction, the lower right direction, the upper left direction, and the lower left direction, the feature point comparison unit 76 determines the characteristic points according to the movement direction of the flying object 10 . The relationship between a first image area 98 and a second image area 102 against which points are compared is shown.

For example, when the moving direction of the flying object 10 is the upper right direction, the feature point comparison unit 76 sets the first divided region 96-2, the first divided region 96-3, and the first divided region 96-2 as the first image region 98. 96-6, and the second feature points included in the second divided regions 100-4, 100-7, and 100-8 as the second image region 102 Compare with Further, when the moving direction of the flying object 10 is the lower right direction, the feature point comparison unit 76 selects the first divided region 96-6, the first divided region 96-8, and the first divided region 96-6 as the first image region 98. A first feature point included in the region 96-9 and a second feature included in the second divided regions 100-1, 100-2, and 100-4 as the second image region 102 Compare points.

Further, when the moving direction of the flying object 10 is the upper left direction, the feature point comparison unit 76 sets the first divided region 96-1, the first divided region 96-2, and the first divided region 96-1 as the first image region 98. 96-4, and the second feature points included in the second divided regions 100-6, 100-8, and 100-9 as the second image region 102 Compare with

Further, when the moving direction of the flying object 10 is the lower left direction, the feature point comparison unit 76 selects the first segmented region 96-4, the first segmented region 96-7, and the first segmented region 96-4 as the first image region 98. 96-8, and the second feature points included in the second divided regions 100-2, 100-3, and 100-6 as the second image region 102 Compare with Thus, the first image region 98 and the second image region 102 whose feature points are compared by the feature point comparison unit 76 are changed according to the moving direction of the flying object 10 .

Next, the operation of the flight imaging device 1 according to this embodiment will be described with reference to FIG. FIG. 18 shows an example of the flow of imaging processing according to this embodiment.

In the imaging process shown in FIG. 18, first, in step ST10, the first imaging control unit 62 causes the image sensor 36 to image the imaging target area 3A (see FIG. 4). Image data for synthesis is obtained by imaging the imaging target region 3A with the image sensor 36 under the control of the first imaging control section 62 . After the process of step ST10 is executed, the imaging process proceeds to step ST12.

In step ST12, the first feature point information generation unit 64 generates first feature points included in the entire area of the image for synthesis 92 for each first divided area 96 based on the image data for synthesis acquired in step ST10. First feature point information indicating the coordinates of the first feature point for each of the extracted first divided areas 96 is generated (see FIG. 4). After the process of step ST12 is executed, the imaging process proceeds to step ST14.

At step ST14, the first movement determination unit 66 determines whether or not the flying object 10 is moving (see FIG. 5). In step ST14, if the flying object 10 is moving, the determination is affirmative, and the imaging process proceeds to step ST16. In step ST14, if the flying object 10 is not moving, the determination is negative, and the imaging process is executed again in step ST14.

At step ST16, the movement direction information generation unit 68 acquires the movement direction of the flying object 10 and generates movement direction information indicating the acquired movement direction (see FIG. 5). After the process of step ST16 is executed, the imaging process proceeds to step ST18.

At step ST18, the second imaging control unit 70 causes the image sensor 36 to image the imaging target region 3A (see FIG. 6). Display image data is obtained by capturing an image of the image capturing target area 3A with the image sensor 36 under the control of the second image capturing control section 70 . After the process of step ST18 is executed, the imaging process proceeds to step ST20.

At step ST20, the display control unit 72 causes the display device 24 to display the display image 94 (that is, the live view image) based on the display image data obtained at step ST18 (see FIG. 6). After the process of step ST20 is executed, the imaging process proceeds to step ST22.

In step ST22, the second feature point information generation unit 74 generates the second image region of the display image 94 based on the movement direction information obtained in step ST16 and the display image data obtained in step ST18. Second feature point information indicating the coordinates of the second feature points for every 102 is generated (see FIG. 6). After the process of step ST22 is executed, the imaging process proceeds to step ST24.

In step ST24, the feature point comparison unit 76 calculates the first image region of the image for synthesis 92 based on the first feature point information generated in step ST12 and the second feature point information generated in step ST22. 98 and the second feature points included in the second image area 102 of the display image 94 are compared, and comparison result information indicating the comparison result is generated (see FIG. 7). . The above description is for the case where the process of step ST24 is the first process. If the process of step ST24 is the second or subsequent process, the feature point comparison unit 76 compares the first feature point information generated in step ST32 or step ST34 described later and the second feature point information generated in step ST22. Based on the point information, the first feature point and the second feature point are compared, and comparison result information indicating the comparison result is generated (see FIG. 11 or FIG. 14). After the process of step ST24 is executed, the imaging process proceeds to step ST26.

In step ST26, the overlap determination unit 78 acquires the comparison result information generated in step ST24, and based on the acquired comparison result information, the overlap ratio between the image for synthesis 92 and the image for display 94 is set to the default value. It is determined whether or not it is equal to or less than the threshold (see FIG. 8). In step ST26, if the overlap rate between the image for synthesis 92 and the image for display 94 is equal to or less than the predetermined threshold value, the determination is affirmative, and the imaging process proceeds to step ST28. In step ST26, if the overlap ratio between the image for synthesis 92 and the image for display 94 exceeds the predetermined threshold value, the determination is negative, and the imaging process proceeds to step ST16.

At step ST28, the first imaging control unit 62 causes the image sensor 36 to image the imaging target region 3A (see FIG. 9 or FIG. 12). Image data for synthesis is obtained by imaging the imaging target region 3A with the image sensor 36 under the control of the first imaging control section 62 . After the process of step ST28 is executed, the imaging process proceeds to step ST30.

In step ST30, the second movement determination unit 80 determines whether the flying object 10 continues to move based on the movement instruction signal received by the imaging device 30 and the movement direction information generated in step ST16. is determined (see FIG. 10 or FIG. 13). In step ST30, if the flying object 10 continues to move, the determination is affirmative, and the imaging process proceeds to step ST32. In step ST30, if the flying object 10 has not continued to move, that is, if the flying object 10 has stopped moving, the determination is negative, and the imaging process proceeds to step ST34.

In step ST32, the first feature point information generation unit 64 generates the first image region of the image for synthesis 92 based on the moving direction information obtained in step ST16 and the image data for synthesis obtained in step ST28. The first feature point information indicating the coordinates of the first feature points for every 98 is generated (see FIG. 10). After the process of step ST32 is executed, the imaging process proceeds to step ST36.

In step ST34, the first feature point information generation unit 64 generates first feature points included in the entire area of the image for synthesis 92 for each first divided area 96 based on the image data for synthesis acquired in step ST28. First feature point information indicating the coordinates of the first feature point for each of the extracted first divided areas 96 is generated (see FIG. 13). After the process of step ST34 is executed, the imaging process proceeds to step ST36.

At step ST36, the processor 46 determines whether or not a condition for terminating the imaging process (end condition) is met. Examples of end conditions include a condition that the user has given an instruction to end the imaging process to the imaging device 30, or a condition that the number of images for synthesis 92 has reached the number specified by the user. In step ST36, if the termination condition is not met, the determination is negative, and the imaging process proceeds to step ST14. In step ST36, if the end condition is met, the determination is affirmative and the imaging process ends. Note that the imaging method described as the operation of the flight imaging device 1 described above is an example of the “imaging method” according to the technology of the present disclosure.

As described above, in the flight imaging device 1 according to the present embodiment, the processor 46 compares the first feature point included in the image for synthesis 92 and the second feature point included in the image for display 94, and the result is Based on this, the overlap ratio between the image for synthesis 92 and the image for display 94 is determined. Here, when the processor 46 determines the overlap ratio, the first image region 98, which is the image region included in the image for synthesis 92 and for obtaining the first feature point, and the A second image area 102, which is an image area included in the image 94 and for obtaining the second feature point, is changed according to the moving direction. Therefore, for example, based on the result of comparing the first feature points included in the entire area of the image for synthesis 92 and the second feature points included in the entire area of the image for display 94, the image for synthesis 92 and the image for display 94, the load required for the processing of the processor 46 when determining the overlap ratio can be reduced.

The processor 46 also compared the first feature points included in the first image area 98 of the image for synthesis 92 with the second feature points included in the second image area 102 of the image for display 94. Based on the results, the overlap ratio between the image for synthesis 92 and the image for display 94 is determined. Therefore, for example, based on the result of comparing the first feature points included in the entire area of the image for synthesis 92 and the second feature points included in the entire area of the image for display 94, the image for synthesis 92 and the image for display As in the case of judging the overlap rate with 94, it is possible to ensure the judgment accuracy of the overlap rate.

Also, the processor 46 causes the display device 24 to display the display image 94 . Therefore, based on the display image 94, the position of the imaging target area 3A can be confirmed.

Also, the first image area 98 is a partial image area of the synthesis image 92 , and the second image area 102 is a partial image area of the display image 94 . Therefore, for example, compared with the case of comparing the first feature points included in the entire area of the first image and the second feature points included in the entire area of the second image, the first feature points and the second feature points It is possible to reduce the load required for the processing of the processor 46 when comparing the .

Also, the first image area 98 is an area located on the front side in the movement direction of the image for synthesis 92, and the second image area 102 is an area located on the rear side of the image for display 94 in the movement direction. Therefore, by comparing the first feature points included in the first image area 98 and the second feature points included in the second image area 102, the overlap rate between the synthesis image 92 and the display image 94 can be determined. can judge.

Also, the processor 46 selects a first image region 98 from a plurality of first divided regions 96 obtained by dividing the composite image 92 . Therefore, for example, compared to setting the first image region 98 from the image for synthesis 92 in a state in which the image for synthesis 92 is not divided into a plurality of first divided regions 96, the first image region 98 can be set by simple processing. Can be set. Similarly, the processor 46 selects a second image region 102 from a plurality of second divided regions 100 obtained by dividing the display image 94 . Therefore, for example, compared to setting the second image area 102 from the display image 94 in a state in which the display image 94 is not divided into a plurality of second divided areas 100, the second image area 102 can be set by simple processing. Can be set.

In addition, the first image region 98 includes three first segmented regions 96 out of the plurality of first segmented regions 96, and the second image region 102 includes three second segmented regions 100 out of the plurality of second segmented regions 100. A divided area 100 is included. Therefore, for example, the first image region 98 includes less than three first segmented regions 96 out of the plurality of first segmented regions 96, and the second image region 102 includes three out of the plurality of second segmented regions 100. It is possible to improve the determination accuracy of the overlap rate compared to the case of including less than the second divided areas 100 .

Also, the first image area 98 and the second image area 102 are each rectangular areas. Therefore, for example, the first feature point and the second feature point can be compared with a simpler process than when the first image area 98 and the second image area 102 are areas with shapes other than rectangles. be able to.

Also, the first feature point is a feature point included only in the first image area 98 , and the second feature point is a feature point included only in the second image area 102 . Therefore, for example, the overlap ratio It is possible to reduce the load required for the processing of the processor 46 when determining

Also, when the flying object 10 is moving, the processor 46 extracts first feature points included in a first image area 98 that is a partial image area of the display image 94 . Therefore, for example, when the moving body is moving, the processor 46 extracts the first feature points included in the entire area of the display image 94, and the processor 46 extracts the first feature points. 46 processing load can be reduced.

Also, the processor 46 extracts feature points included in the entire area of the display image 94 when the flying object 10 is stationary. Therefore, the first feature point included in the first image area 98 of the display image 94 can be extracted regardless of the direction in which the flying object 10 moves from the stationary state.

In addition, the area of the first image area 98 and the area of the second image area 102 are each larger than the overlapping area (hereinafter simply referred to as "overlap area") when the adjacent images for synthesis 92 are synthesized. Set in a large area. Therefore, for example, compared to the case where the area of the first image area 98 and the area of the second image area 102 are set to areas smaller than the overlap area, the first feature point and the second feature point Based on this, it is possible to secure determination accuracy when determining the overlap ratio between the image for synthesis 92 and the image for display 94 .

Also, the imaging device 30 is mounted on the aircraft 10 . Therefore, the imaging device 30 can image the imaging target region 3A while the aircraft 10 is flying.

Note that, in the above embodiment, the first image region 98 includes three first divided regions 96 out of the plurality of first divided regions 96, and the second image region 102 out of the plurality of second divided regions 100. includes three second divided regions 100 of However, as shown in FIG. 19 as an example, the first image region 98 is one of the first divided regions 96 of the plurality of first divided regions 96, and the second image region 102 is the plurality of second divided regions. It may be one of the second divided regions 100 of the regions 100 . Then, the feature point comparison unit 76 compares the first feature points included in the first image region 98 that is one first divided region 96 and the second feature points included in the second image region 102 that is one second divided region 100 . Two feature points may be compared and comparison result information indicating the comparison result may be generated.

According to the example shown in FIG. 19 , for example, the feature point comparison unit 76 compares the first feature points included in the first image area 98 that is the three first segmented areas 96 and the three second segmented areas 100. Compared to the case of comparing the second feature points included in the second image area 102, the load required for the processing of the processor 46 when comparing the first feature points and the second feature points can be reduced. .

Also, although not shown, the first image region 98 is two of the first divided regions 96 of the plurality of first divided regions 96, and the second image region 102 is of the plurality of second divided regions 100. may be two second segmented regions 100. Then, the feature point comparison unit 76 compares the first feature points included in the first image regions 98 that are the two first divided regions 96 and the second feature points that are included in the second image regions 102 that are the two second divided regions 100 . Two feature points may be compared and comparison result information indicating the comparison result may be generated.

According to this example as well, for example, the feature point comparison unit 76 compares the first feature points included in the first image regions 98, which are the three first divided regions 96, and the second image regions, which are the three second divided regions 100. Compared to the case of comparing the second feature points included in 102, the load required for the processing of the processor 46 when comparing the first feature points and the second feature points can be reduced.

Note that the first image region 98 includes four or more first divided regions 96 out of the plurality of first divided regions 96, and the second image region 102 includes four out of the plurality of second divided regions 100. The second segmented region 100 described above may be included.

Also, in the above embodiment, the first image area 98 and the second image area 102 are each rectangular areas. However, as shown in FIG. 20 as an example, when the moving direction of the flying object 10 is an inclined direction (that is, a direction inclined with respect to the horizontal or vertical direction of the display image 94), the processor 46 The first image area 98 and the second image area 102 may be set as follows. That is, the processor 46 selects one first triangular area 104 out of the plurality of first triangular areas 104 including the four corners of the image for synthesis 92 (that is, the moving direction of the four corners of the image for synthesis 92). A first triangular region 104 ) containing the forwardly located corner may be set to the first image region 98 . Also, the processor 46 selects one of the plurality of second triangular regions 106 including the four corners of the display image 94 (that is, the movement direction of the four corners of the display image 94). A second triangular area 106 ) containing the corner located on the rear side may be set in the second image area 102 .

According to the example shown in FIG. 20 as well, the feature point comparison unit 76 compares the first feature points included in the first image area 98 and the second feature points included in the second image area 102, whereby the comparison is performed. Result information can be generated.

Also, in the above embodiment, the second imaging control unit 70 causes the entire area of the image sensor 36 to output display image data. However, as shown in FIG. 21 as an example, the second imaging control unit 70 acquires the movement direction of the flying object 10 based on the movement instruction signal, and uses the cropping function of the image sensor 36 to obtain the image sensor 36 The image data for display may be output only from the area 36A located on the front side in the moving direction among all the pixels of . That is, when acquiring the display image 94, the second imaging control unit 70 causes the image sensor 36 to output only image data used for comparing the first feature points and the second feature points. good too. Then, the second feature point information generation unit 74 generates the second feature point information of the display image 94 based on the display image data output only from the region 36A positioned on the front side in the movement direction of the entire region of the image sensor 36. Second feature point information regarding the two-image region 102 may be generated.

According to the example shown in FIG. 21, the second feature point information regarding the second image area 102 of the display image 94 is higher than, for example, the case where the display image data is output from the entire area of the image sensor 36 . It is possible to reduce the load required for the processing of the processor 46 when generating .

Note that the processor 46 is used to compare the first feature points and the second feature points in the display image data output from the entire area of the image sensor 36 when acquiring the display image 94. Only image data may be acquired. According to this example, compared to the case where the second feature point information about the entire area of the display image 94 is generated based on the display image data output from the entire area of the image sensor 36, the second feature points The load required for the processing of the processor 46 when generating information can be reduced.

Further, as shown in FIGS. 22 and 23 as an example, the first feature point information generation unit 64 changes the area of the first image region 98 in the synthesis image 92 according to the moving speed of the aircraft 10. You may Similarly, the second feature point information generator 74 may change the area of the second image area 102 in the display image 94 according to the moving speed of the flying object 10 .

Specifically, in the example shown in FIG. 22, the first feature point information generator 64 increases the area of the first image region 98 as the moving speed of the flying object 10 increases. Similarly, the second feature point information generator 74 increases the area of the second image area 102 as the moving speed of the flying object 10 increases. According to the example shown in FIG. 22, for example, even if the moving speed of the flying object 10 increases, the overlap area is smaller than that in the case where the area of the first image area 98 and the area of the first image area 98 are constant. Rate determination accuracy can be improved.

On the other hand, in the example shown in FIG. 23, the first feature point information generator 64 reduces the area of the first image region 98 as the moving speed of the flying object 10 increases. Similarly, the second feature point information generator 74 reduces the area of the second image area 102 as the moving speed of the flying object 10 increases. According to the example shown in FIG. 23, for example, even if the moving speed of the flying object 10 increases, the overlap ratio It is possible to reduce the load required for the processing of the processor 46 when determining

22 and 23, both the area of the first image area 98 and the area of the second image area 102 are changed, but the area of the first image area 98 and the area of the second image area 102 are changed. Only one of the areas may be changed.

Further, in the above embodiment, when judging the overlap rate, the processor 46 determines the first image area 98 included in the image for synthesis 92 and the second image area 102 included in the image for display 94 according to the movement direction. to change. However, when determining the overlap ratio, the processor 46 selects only one of the first image area 98 included in the image for synthesis 92 and the second image area 102 included in the image for display 94 in the moving direction. may be changed accordingly.

Further, in the above embodiment, the number of the plurality of first divided regions 96 is nine as an example, but the number of the plurality of first divided regions 96 may be other than nine (eg, four, six, eight, etc.). 1, or 16, etc.). Similarly, the number of the plurality of second segmented regions 100 is nine as an example, but the number of the plurality of second segmented regions 100 may be other than nine (eg, four, six, eight, or sixteen). etc.) is also acceptable. Also, the number of the plurality of first segmented regions 96 and the number of the plurality of second segmented regions 100 may be different.

Also, in the above embodiment, the movement direction information generation unit 68 acquires the movement instruction signal transmitted from the transmitter 20 and generates movement direction information based on the acquired movement instruction signal. However, the movement direction information generator 68 may, for example, acquire a movement signal generated by the aircraft 10 and generate movement direction information based on the acquired movement signal. Further, the movement direction information generation unit 68 may generate movement direction information based on information acquired from the imaging device 30 and/or various devices mounted on the aircraft 10 . Various devices include, for example, satellite positioning systems (eg, global positioning systems), accelerometers, three-dimensional sensors (eg, LiDAR or stereo cameras), and the like.

Further, in the above-described embodiment, an example in which the imaging device 30 is mounted on the flying object 10 is given, but the imaging device 30 can be mounted on various mobile objects (for example, a gondola, an automatic guided robot, an unmanned guided vehicle, or a high-speed aircraft). inspection vehicle) or the like.

Also, in the above embodiment, the image for synthesis 92 is acquired by the imaging device 30, but the image for synthesis 92 may be acquired by an imaging device other than the imaging device 30. Then, the overlap rate between the synthesis image 92 acquired by another imaging device and the display image 94 acquired by the imaging device 30 may be determined.

Further, in the above embodiment, the overlap ratio between the live view image as the display image 94 and the composition image 92 is determined. A rate may be determined.

Further, in the above-described embodiment, the overlap ratio between the image for synthesis 92 and the image for display 94 is determined. A rate may be determined.

In addition, although the processor 46 is illustrated in each of the above embodiments, at least one other CPU, at least one GPU, and/or at least one TPU may be used in place of or together with the processor 46 can be

Further, in each of the above-described embodiments, an example of a form in which the imaging program 60 is stored in the storage 48 has been described, but the technology of the present disclosure is not limited to this. For example, the imaging program 60 may be stored in a portable non-temporary computer-readable storage medium such as an SSD or USB memory (hereinafter simply referred to as "non-temporary storage medium"). An imaging program 60 stored in a non-temporary storage medium is installed in the computer 32 of the imaging device 30 , and the processor 46 executes processing according to the imaging program 60 .

Also, the imaging program 60 is stored in another computer connected to the imaging device 30 via a network or in a storage device such as a server device, and the imaging program 60 is downloaded in response to a request from the imaging device 30 and stored in the computer 32 . may be installed in

Further, it is not necessary to store all of the imaging program 60 in a storage device such as another computer or server device connected to the imaging device 30, or in the storage 48, and a part of the imaging program 60 may be stored. good too.

In addition, although the computer 32 is built into the imaging device 30, the technology of the present disclosure is not limited to this, and the computer 32 may be provided outside the imaging device 30, for example.

Further, although the computer 32 including the processor 46, the storage 48, and the RAM 50 is illustrated in each of the above-described embodiments, the technology of the present disclosure is not limited to this, and instead of the computer 32, ASIC, FPGA, and/or Or you may apply the device containing PLD. Also, instead of the computer 32, a combination of hardware configuration and software configuration may be used.

Also, the following various processors can be used as hardware resources for executing the various processes described in each of the above embodiments. Examples of processors include CPUs, which are general-purpose processors that function as hardware resources that execute various processes by executing software, that is, programs. Also, processors include, for example, dedicated electronic circuits such as FPGAs, PLDs, and ASICs, which are processors having circuit configurations specially designed to execute specific processing. A memory is built in or connected to each processor, and each processor uses the memory to perform various processes.

Hardware resources that perform various processes may be configured with one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or CPUs). and FPGA). Also, the hardware resource for executing various processes may be one processor.

As an example of configuration with one processor, first, there is a form in which one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Secondly, as typified by SoC, etc., there is a mode of using a processor that implements the function of the entire system including a plurality of hardware resources for executing various processes with a single IC chip. In this way, various processes are realized using one or more of the above various processors as hardware resources.

Furthermore, as the hardware structure of these various processors, more specifically, an electronic circuit in which circuit elements such as semiconductor elements are combined can be used. Also, the line-of-sight detection process described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps added, and the order of processing may be changed without departing from the scope of the invention.

The descriptions and illustrations shown above are detailed descriptions of the parts related to the technology of the present disclosure, and are merely examples of the technology of the present disclosure. For example, the above descriptions of configurations, functions, actions, and effects are descriptions of examples of configurations, functions, actions, and effects of portions related to the technology of the present disclosure. Therefore, unnecessary parts may be deleted, new elements added, or replaced with respect to the above-described description and illustration without departing from the gist of the technology of the present disclosure. Needless to say. In addition, in order to avoid complication and facilitate understanding of the portion related to the technology of the present disclosure, the descriptions and illustrations shown above require particular explanation in order to enable implementation of the technology of the present disclosure. Descriptions of common technical knowledge, etc., that are not used are omitted.

In this specification, "A and/or B" is synonymous with "at least one of A and B." That is, "A and/or B" means that only A, only B, or a combination of A and B may be used. Also, in this specification, when three or more matters are expressed by connecting with "and/or", the same idea as "A and/or B" is applied.

All publications, patent applications and technical standards mentioned herein are expressly incorporated herein by reference to the same extent as if each individual publication, patent application and technical standard were specifically and individually noted to be incorporated by reference. incorporated by reference into the book.

Claims

An imaging device comprising a processor,
The processor
obtain a first image;
Acquiring a moving direction of a moving body on which the imaging device is mounted,
Acquiring a second image obtained by being imaged by the imaging device when the moving object moves,
Determining an overlap rate between the first image and the second image based on the result of comparing the first feature points included in the first image and the second feature points included in the second image,
When determining the overlap rate, an image area included in the first image, which is an image area for obtaining the first feature point, and/or included in the second image an imaging device that changes a second image area, which is an image area for obtaining the second feature point, according to the moving direction.
The imaging device according to claim 1, wherein the processor causes a display device to display the second image as a display image.
The first image area is a partial image area of the first image,
The imaging device according to claim 1 or 2, wherein the second image area is a partial image area of the second image.
The first image area is an area located on the front side of the movement direction in the first image,
The imaging device according to any one of claims 1 to 3, wherein the second image area is an area located on the rear side in the moving direction in the second image.
The processor
selecting the first image region from a plurality of first divided regions obtained by dividing the first image;
The imaging device according to any one of claims 1 to 4, wherein the second image area is selected from a plurality of second divided areas obtained by dividing the second image.
The first image region includes two or more first segmented regions of the plurality of first segmented regions,
The imaging device according to claim 5, wherein the second image area includes two or more second divided areas among the plurality of second divided areas.
the first image region is one first segmented region of the plurality of first segmented regions;
The imaging device according to claim 5, wherein the second image area is one of the plurality of second divided areas.
The imaging device according to any one of claims 1 to 7, wherein the first image area and the second image area are rectangular areas.
The processor includes a corner located on the front side of the moving direction out of the four corners of the first image when the moving direction is a direction that is inclined with respect to the horizontal direction or the vertical direction of the second image. A first triangular area is set as the first image area, and a second triangular area including a corner located on the rear side of the moving direction among the four corners of the second image is set as the second image area. The imaging device according to any one of claims 1 to 7.
10. The processor of any one of claims 1 to 9, wherein when obtaining the second image, the processor obtains from an image sensor only image data used to compare the first feature points and the second feature points. The imaging device according to any one of the items.
The first feature point is a feature point included only in the first image area,
The imaging apparatus according to any one of claims 1 to 10, wherein the second feature point is a feature point included only in the second image area.
The imaging according to any one of claims 1 to 11, wherein the processor changes the area of the first image area and/or the area of the second image area according to the moving speed of the moving object. Device.
13. The imaging apparatus according to claim 12, wherein the processor increases the area of the first image area and/or the area of the second image area as the moving speed of the moving object increases.
13. The imaging apparatus according to claim 12, wherein the processor reduces the area of the first image area and/or the area of the second image area as the moving speed of the moving body increases.
The processor extracts the first feature points included in the first image area, which is a partial image area of the first image, when the moving body is moving. 15. The imaging device according to any one of 14.
The imaging apparatus according to any one of claims 1 to 15, wherein the processor extracts feature points included in the entire area of the first image when the moving body is stationary.
2. The area of the first image area and the area of the second image area are each set to a larger area than an overlap area, which is an area where the adjacent first images overlap when combined. 17. The imaging device according to any one of claims 16 to 16.
The imaging apparatus according to any one of claims 1 to 17, wherein the mobile object is an aircraft.
obtaining a first image;
Acquiring the direction of movement of a moving object equipped with an imaging device;
Acquiring a second image captured by the imaging device when the moving body moves;
Determining an overlap ratio between the first image and the second image based on a result of comparing the first feature points included in the first image and the second feature points included in the second image. ,as well as,
When determining the overlap rate, an image area included in the first image, which is an image area for obtaining the first feature point, and/or included in the second image changing a second image area, which is an image area for obtaining the second feature point, according to the moving direction.
obtaining a first image;
Acquiring the direction of movement of a moving object equipped with an imaging device;
Acquiring a second image captured by the imaging device when the moving body moves;
Determining an overlap rate between the first image and the second image based on a result of comparing the first feature points included in the first image and the second feature points included in the second image. ,as well as,
When determining the overlap rate, an image area included in the first image, which is an image area for obtaining the first feature point, and/or included in the second image A program for causing a computer to execute a process including changing a second image area, which is an image area for obtaining the second feature points, according to the moving direction.