WO2025258022A1

WO2025258022A1 - Coregistering apparatus, coregistering method, and non-transitory computer-readable medium

Info

Publication number: WO2025258022A1
Application number: PCT/JP2024/021500
Authority: WO
Inventors: Tsenjung Tai
Original assignee: NEC Corp
Current assignee: NEC Corp
Priority date: 2024-06-13
Filing date: 2024-06-13
Publication date: 2025-12-18
Anticipated expiration: 2026-12-13

Abstract

A coregistering apparatus performs: acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other; performing, for each one of two or more different amounts of shift to be applied to the second captured image: shifting the second captured image; detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and determine, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.

Description

COREGISTERING APPARATUS, COREGISTERING METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

　　The present disclosure generally relates to coregistering apparatus, coregistering method, and non-transitory computer-readable medium.

　　Techniques to coregister two or more images has been developed. PTL 1 disclose a technique to coregister two SAR (synthetic-aperture radar) images using SIFT (Scale-Invariant Feature Transform) key-points of the images.

PTL1: Chinese Patent Publication CN103839265A

　　It is possible that two images to be coregistered each other are images in which the same place are captured from different angles from each other. PTL1 does not clearly address this fact. An objective of the present disclosure is to provide a novel technique to coregister images in which the same place is captured from different angles from each other.

　　The present disclosure provides a coregistering apparatus that comprises at least one memory that is configured to store instructions and at least one processor that is configured to execute the instructions to: acquire a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other; perform, for each one of two or more different amounts of shift to be applied to the second captured image: shifting the second captured image; detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and determine, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.

　　The present disclosure further provides a coregistering method that is performed by a computer, comprises: acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other; performing, for each one of two or more different amounts of shift to be applied to the second captured image: shifting the second captured image; detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.

　　The present disclosure further provides a non-transitory computer readable storage medium storing a program. The program that causes a computer to execute: acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other; performing, for each one of two or more different amounts of shift to be applied to the second captured image: shifting the second captured image; detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.

　　According to the present disclosure, a novel technique to coregister images in which the same place is captured from different angles from each other is provided.

Fig. 1 illustrates an overview of a coregistering apparatus. Fig. 2 is a block diagram showing an example of the functional configuration of the coregistering apparatus. Fig. 3 is a block diagram illustrating an example of the hardware configuration of a computer realizing the coregistering apparatus. Fig. 4 shows a flowchart illustrating an example flow of processes performed by the coregistering apparatus. Fig. 5 illustrates an example way of detecting the common low-entropy regions.

　　Example embodiments according to the present disclosure will be described hereinafter with reference to the drawings. The same reference signs are assigned to the same elements throughout the drawings, and redundant explanations are omitted as necessary. In addition, predetermined information (e.g., a predetermined value or a predetermined threshold) is stored in advance in a storage unit to which a computer using that information has access unless otherwise described. In the present disclosure, a storage unit may be implemented with one or more storage devices, such as hard disks, solid-state drives (SSDs), or random-access memories (RAMs).

FIRST EXAMPLE EMBODIMENT
<Overview>
　　Fig. 1 illustrates an overview of a coregistering apparatus 2000. It is noted that Fig. 1 does not limit operations of the coregistering apparatus 2000, but merely show an example of possible operations of the coregistering apparatus 2000.

　　The coregistering apparatus 2000 is configured to perform coregistration of a first captured image 10 and a second captured image 20. The first captured image 10 is generated by a sensor 50 that captures a target place 30 from a first angle. The second captured image 20 is generated by a sensor 60 that captures the target place 30 from a second angle. It is noted that the sensor 60 may be a different sensor from the sensor 50, or may be the same sensor as the sensor 50. In the latter case, the sensor 60 is the sensor 50 whose capturing angle (or antenna) is adjusted after generating the first captured image 10.

　　Due to the difference of the angle of the sensor 50 and the angle of the sensor 60, the appearance of the target place 30 in the first captured image 10 and the appearance of the target place 30 in the second captured image 20 may be different from each other. This fact makes the coregistration of the first captured image 10 and the second captured image 20 more difficult than the coregistration of two images in which the target place 30 has the same appearance as each other.

　　To tackle this issue, the coregistering apparatus 2000 coregisters the first captured image 10 and the second captured image 20 based on a part of the first captured image 10 and a part of the second captured image 20 that are expected to be not significantly affected by the difference in capturing angles. Specifically, the coregistering apparatus 2000 works as follow.

　　For each one of two or more different amounts of shift to be applied to the second captured image 20, the coregistering apparatus 2000 performs a process referred to as "similarity computing process". The similarity computing process includes following processes. First, the coregistering apparatus 2000 shifts the second captured image 20. Then, the coregistering apparatus 2000 detects one or more regions, referred to as "common low-entropy regions," from the first captured image 10 and the second captured image 20 after shifting. Hereinafter, the second captured image 20 after shifting is referred to as the "shifted second captured image 20" or the "shifted image." The common low-entropy region is a region whose entropy is lower than a pre-defined threshold (hereinafter, referred to as "entropy threshold") in both the first captured image 10 and the shifted image. The coregistering apparatus 2000 computes a degree of similarity between the common low-entropy regions in the first captured image 10 and the common low-entropy regions in the shifted image.

　　After performing the similarity computing process for each one of the two or more different amounts of shift, the coregistering apparatus 2000 determines the amount of shift with which the first captured image 10 and the second captured image 20 are coregistered each other. This determination is performed based on the degree of similarity computed for each one of the two or more different amounts of shift.

　　It is noted that the first captured image 10 may be an optical image or a radar image. When the first captured image 10 is an optical image, the sensor 50 is an optical camera that is configured to receive light to generate an optical image based on the received light. When the first captured image 10 is a radar image, the sensor 50 is a radar that is configured to transmit radio waves, receive reflection of the radio waves, and generate a radar image based on the received reflection of the radio waves. The sensor 50 may be installed on an artificial satellite to capture objects on the Earth, other planets, satellites, etc. An example of the radar used as the sensor 50 is a synthetic-aperture radar.

　　Similarly, the second captured image 20 may be an optical image or a radar image. When the second captured image 20 is an optical image, the sensor 60 is an optical camera that is configured to receive light to generate an optical image based on the received light. When the second captured image 20 is a radar image, the sensor 60 is a radar that is configured to transmit radio waves, receive reflection of the radio waves, and generate a radar image based on the received reflection of the radio waves. The sensor 60 may be installed on an artificial satellite to capture objects on the Earth, other planets, satellites, etc. An example of the radar used as the sensor 60 is a synthetic-aperture radar.

<Example of Advantageous Effect>
　　When two images to be coregistered are generated by capturing the same place from different angles from each other, the appearances of that place in each image may differ. Specifically, in the images, the appearance of taller objects (e.g., buildings) may be significantly affected by the sensor's capturing angle than the appearance of shorter objects (e.g., roads or fields). Therefore, it is preferable to exclude the regions containing these taller objects when coregistering the images.

　　Regarding these taller objects, it is expected that they have higher entropy than shorter objects in the images. For example, buildings in SAR images may exhibit higher entropy due to their complex features, including bright patterns resulting from their geometrical and radiometric properties, such as double bounce effects and layovers.

　　Based on the above-mentioned insights, the coregistering apparatus 2000 is configured to compare the common low-entropy regions of the first captured image 10 and the common low-entropy regions of the second captured image 20 after shifting for coregistration of them. By doing so, the coregistering apparatus 2000 can reduce the influence of differences in the capturing angles of the sensor 50 and the sensor 60, thereby achieving accurate coregistration of the first captured image 10 and the second captured image 20.

　　The coregistering apparatus 2000 is useful in and applicable to various situations. One of the situations in which the coregistering apparatus 2000 is useful is the situation where a disaster, such as an earthquake, has occurred. In this situation, it is preferable to ascertain the damage caused by the disaster as quickly as possible to handle the problems caused by the disaster (e.g., providing appropriate supports to the victims).

　　To ascertain the damage caused by the disaster, it is effective to detect change in the disaster-stricken area by comparing the images obtained from the satellites that capture that area, such as SAR images. Specifically, for each of the sections in the disaster-stricken area, the change in the section can be detected by comparing the image generated by capturing that section before the disaster and the image generated by capturing that section after the disaster.

　　Suppose that the images obtained from the same angle by the same satellite are used. In this case, it takes time (e.g., ten days) to obtain the images to be compared for the change detection since the satellite keeps moving. Thus, in order to quickly ascertain the damage in the disaster-stricken area, it is preferable to use the images obtained from different angles by adjusting the antennas of the same satellite or from different satellites that happen to fly close to the area of interest.

　　As a preprocess for change detection in the images obtained from different angles, it is necessary to coregister those images. As mentioned above, the images obtained from different angles may include the same place with different appearances, and this fact makes coregistration of these images difficult. Therefore, it is effective to apply the coregistering apparatus 2000 to this situation since the coregistering apparatus 2000 can accurately coregister two images obtained from different angles.

　　Hereinafter, more detailed explanation of the coregistering apparatus 2000 will be described.

<Example of Functional Configuration>
　　Fig. 2 is a block diagram showing an example of the functional configuration of the coregistering apparatus 2000. The coregistering apparatus 2000 includes an acquiring unit 2020, a similarity computing unit 2040, and a determining unit 2060. The acquiring unit 2020 acquires the first captured image 10 and the second captured image 20. The similarity computing unit 2040 performs the similarity computing process for each one of two or more different amounts of shift to be applied to the second captured image 20.

　　In the similarity computing process, the similarity computing unit 2040 works as follows. First, the similarity computing unit 2040 shifts the second captured image 20. Next, the similarity computing unit 2040 detects one or more common low-entropy regions from the first captured image 10 and the shifted image. Then, the similarity computing unit 2040 computes a degree of similarity between the common low-entropy regions in the first captured image 10 and the common low-entropy regions in the shifted image.

　　Based on the degree of similarity computed for each amount of shift, the determining unit 2060 determines the amount of shift with which the first captured image 10 and the second captured image 20 are coregistered each other.

<Example of Hardware Configuration>
　　The coregistering apparatus 2000 may be realized by one or more computers. Each of the one or more computers may be a special-purpose computer manufactured for implementing the coregistering apparatus 2000, or may be a general-purpose computer like a personal computer (PC), a server machine, or a mobile device.

　　The coregistering apparatus 2000 may be realized by installing an application in the computer. The application is implemented with a program that causes the computer to function as the coregistering apparatus 2000. In other words, the program is an implementation of the functional units of the coregistering apparatus 2000. There are various ways to acquire the program. For example, the program can be acquired from a storage medium (such as a DVD (digital versatile disc) or a USB (Universal Serial Bus) memory) in which the program is stored in advance. In another example, the program can be acquired by downloading it from a server machine that manages a storage medium in which the program is stored in advance.

　　Fig. 3 is a block diagram illustrating an example of the hardware configuration of a computer 1000 realizing the coregistering apparatus 2000. In Fig. 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input/output (I/O) interface 1100, and a network interface 1120.

　　The bus 1020 is a data transmission channel in order for the processor 1040, the memory 1060, the storage device 1080, and the I/O interface 1100, and the network interface 1120 to mutually transmit and receive data. The processor 1040 is a processer, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), or a DSP (Digital Signal Processor). The memory 1060 is a primary memory component, such as a RAM or a ROM (Read Only Memory). The storage device 1080 is a secondary memory component, such as a hard disk, an SSD, or a memory card. The I/O interface 1100 is an interface between the computer 1000 and peripheral devices, such as a keyboard, mouse, or display device. The network interface 1120 is an interface between the computer 1000 and a network. The network may be a LAN (Local Area Network) or a WAN (Wide Area Network). The storage device 1080 may store the program mentioned above. The processor 1040 executes the program to realize each functional unit of the coregistering apparatus 2000.

　　The hardware configuration of the computer 1000 is not restricted to that shown in Fig. 3. For example, as mentioned-above, the coregistering apparatus 2000 may be realized by plural computers. In this case, those computers may be connected with each other through the network.

<Flow of Process>
　　Fig. 4 shows a flowchart illustrating an example flow of processes performed by the coregistering apparatus 2000. The acquiring unit 2020 acquires the first captured image 10 and the second captured image 20 (S102). Step S104 to 114 constitutes a repetitive execution of the similarity computing process. The similarity computing process is repeatedly performed until a predefined termination condition is satisfied.

　　In Step S104, the coregistering apparatus 2000 determines whether or not the termination condition is satisfied. When it is determined that the termination condition is satisfied, the coregistering apparatus 2000 performs Step S116 next. On the other hand, when it is determined that the termination condition is not satisfied, the coregistering apparatus 2000 performs Step S106 next.

　　There are various termination conditions that can be employed. Suppose that a set of the amounts of shift to be applied to the second captured image 20 is predefined. In this case, the termination condition may be "the similarity computing process has already been performed for each one of the predefined amounts of shift." In another example, suppose that the number of times the similarity computing process has to be performed is predefined. In this case, the termination condition may be "the similarity computing process has already been performed the predefined number of times."

　　In Step S106, the similarity computing unit 2040 determines the amount of shift to be applied to the second captured image 20. The similarity computing unit 2040 shifts the second captured image 20 by the amount determined in Step S106 (S108). The similarity computing unit 2040 detects the common low-entropy regions from the first captured image 10 and the shifted image (S110). The similarity computing unit 2040 computes a degree of similarity between the common low-entropy regions in the first captured image 10 and the common low-entropy regions in the shifted image (S112).

　　S114 is the end of the similarity computing process. Thus, the coregistering apparatus 2000 computes S104 next.

　　After the repetitive execution of the similarity computing process is terminated, the determining unit 2060 determines the amount of shift with which the first captured image and the second captured image 20 are coregistered each other (S116).

<Acquisition of Images: S102>
　　The acquiring unit 2020 acquires the first captured image 10 and the second captured image 20 (S102). Hereinafter, a set of the first captured image 10 and the second captured image 20 is also called "the set of input data".

　　There are various ways to acquire the set of input data. For example, the set of input data is stored in advance in a storage unit in a manner that the coregistering apparatus 2000 can acquire the set of input data. In this case, the acquiring unit 2020 acquires the set of input data from this storage unit.

　　In another example, the set of input data is input by a user of the coregistering apparatus 2000. In this case, the coregistering apparatus 2000 may prompt the user to input the first captured image 10 and the second captured image 20.

　　The captured images to be acquired as the first captured image 10 and the second captured image 20 may be specified by the user or automatically determined by the coregistering apparatus 2000. In the latter case, for example, the coregistering apparatus 2000 prompts the user to enter a place of interest (i.e., the target place 30) and time of interest. In this case, the target place 30 may be a target for determining whether or not there is substantial change before and after the time of interest. When it is required to ascertain the damage in the disaster-stricken area as mentioned above, a section of the disaster-stricken area and the time at which the disaster occurred are handles as the place of interest and the time of interest, respectively.

　　Suppose that each captured image is stored in a storage unit in association with location information (e.g., global positioning system (GPS) coordinates) that indicates the location of the place captured on the captured image and in association with the identifier of the corresponding satellite. In this case, as the first captured image 10, the acquiring unit 2020 acquires the captured image that meets conditions of: (1) including the place of interest; (2) being generated before the time of interest; and (3) having the latest generation time among the captured images meeting the conditions (1) and (2). In addition, as the second captured image 20, the acquiring unit 2020 acquires the captured image that meets conditions of: (1) including the place of interest; (2) being generated after the time of interest; and (3) having the earliest generation time among the captured images meeting the conditions (1) and (2).

<Adding Shift to Second Captured Image: S106 and S108>
　　The similarity computing unit 2040 determines the amount of shift to be applied to the second captured image 20 (S106), and shifts the second captured image 20 by the determined amount (S108). The second captured image 20 are shifted along X-axis (i.e., horizontally), y-axis (i.e., vertically), or both. Hereinafter, the amount of shift along X-axis and that along y-axis are denoted by dX and dY, respectively.

　　There are various ways to determine the amount of shift to be applied, which can be denoted by (dX, dY). In some implementations, a set of the amounts of shift, i.e., a set of (dX, dY), is predefined. In this case, the similarity computing unit 2040 may choose an amount of shift to be applied to the second captured image 20 from the predefined set randomly or sequentially.

　　In other implementations, the similarity computing unit 2040 may determine the amount of shift to be applied randomly.

　　In other implementations, the similarity computing unit 2040 employs an optimization algorithm, specifically a gradient descent method, to iteratively refine the amount of shift applied to the second captured image 20. The process initiates with the similarity computing unit 2040 setting a preliminary estimate for the amount of shift, denoted as (dX, dY). This initial shift is applied to the second captured image 20, and the similarity computing unit 2040 then computes a baseline degree of similarity, S_(dX, dY), between the low-entropy regions of the first captured image 10 and those of the shifted second captured image 20.

　　The similarity computing unit 2040 next perturbs the amount of shift along the X-axis by ε, which is a small and predefined value, resulting in a new shift (dX+ε, dY). The similarity computing unit 2040 applies this adjusted shift to the second captured image 20 and computes a subsequent degree of similarity, S_(dX+ε, dY).

　　The similarity computing unit 2040 calculates an estimate of the partial derivative of the degree of similarity with respect to the shift in the X-axis according to, for example, the following Equation (1).
Equation 1

　　In Equation (1), ex denotes an estimate of the partial derivative of the degree of similarity with respect to the shift in the X-axis.

　　This calculated derivative informs the direction and magnitude of shift adjustment needed to increase similarity.

　　Similarly, the similarity computing unit 2040 adjusts the shift along the Y-axis by the same ε value to obtain (dX, dY+ε), computes another degree of similarity, S_(dX, dY+ε).

　　Then, the similarity computing unit 2040 calculates an estimate of the partial derivative of the degree of similarity with respect to the shift in the Y-axis according to, for example, the following Equation (2).
Equation 2

　　In Equation (2), ey denotes an estimate of the partial derivative of the degree of similarity with respect to the shift in the Y-axis.

　　Using these derived partial derivatives, the similarity computing unit 2040 updates the shift values according to, for example, the following Equation (3).
Equation 3

　　In Equation (3), dX_updated and dY_updated denote the updated amount of shift in the X-axis and the updated amount of shift in the Y-axis, respectively. R denotes a learning rate, which determines the step size of the update in pursuit of maximizing the degree of similarity. The learning rate may be a predetermined parameter or dynamically adjustable parameter.

　　This process of updating the amount of shift is repeated iteratively. Each iteration refines the amount of shift based on the newly computed degrees of similarity and their respective derivatives. The iterations may continue until the incremental improvements in the degrees of similarity fall below a pre-determined threshold or until a pre-determined number of updates has been reached, thereby optimizing the coregistration of the two images to maximize their similarity.

<Detection of Common Low-Entropy Region: S110>
　　The similarity computing unit 2040 detects the common low-entropy regions from the first captured image 10 and the shifted image (S110). Fig. 5 illustrates an example way of detecting the common low-entropy regions. First, the similarity computing unit 2040 detects one or more low-entropy regions from each of the first captured image 10 and the shifted image 70. The low-entropy region is a region whose entropy is lower than the entropy threshold. In Fig. 5, the diagonal right pattern represents the low-entropy regions detected from the first captured image 10, whereas the diagonal left pattern represents the low-entropy regions detected from the shifted image 70.

　　Then, the similarity computing unit 2040 detects, as the common low-entropy regions, one or more regions each of which is included in both one of the low-entropy regions detected from the first captured image 10 and one of the low-entropy regions detected from the shifted image 70. In Fig. 5, the diagonal cross line pattern represents the common low-entropy regions.

　　Hereinafter, the process of detecting the low-entropy regions is explained using the first captured image 10 as a representative example. First, the similarity computing unit 2040 computes the entropy for each pixel of the first captured image 10. Then, the similarity computing unit 2040 determines whether the entropy of each pixel is lower than the entropy threshold. Based on the result of this determination, the similarity computing unit 2040 generates a low-entropy region mask for the first captured image 10, which represents the low-entropy regions detected from the first captured image 10. In Fig. 5, the reference sign 80 is assigned to the low-entropy region mask of the first captured image 10.

　　In the low-entropy region mask 80, the value of pixels whose entropy is determined to be lower than the entropy threshold is set to one while the value of pixels whose entropy is determined not to be lower than the entropy threshold is set to zero. This means that the low-entropy region mask 80 can visually show the low-entropy regions detected from the first captured image 10.

　　It is noted that the entropy of each pixel of an image can be computed in various ways. For example, the similarity computing unit 2040 computes, for each pixel of the first captured image 10, a histogram of the intensity values of neighboring pixels thereof. The neighboring pixels of a particular pixel are pixels within the region of a predefined size (e.g., 7x7) surrounding it. The similarity computing unit 2040 normalizes the histogram of each pixel to obtain a probability distribution of pixel intensities for each pixel. The similarity computing unit 2040 computes the entropy of each pixel using its corresponding probability distribution.

　　The similarity computing unit 2040 handles the shifted image 70 in the same manner as the first captured image 10, thereby generating the low-entropy region mask of the shifted image 70. In Fig. 5, the reference sign 90 is assigned to the low-entropy region mask of the shifted image 70.

　　The similarity computing unit 2040 generates a common low-entropy region mask, which represents the common low-entropy regions, based on the low-entropy region mask of the first captured image 10 and the low-entropy region mask of the shifted second captured image 20. In Fig. 5, the reference sign 100 is assigned to the common low-entropy region mask. The common low-entropy region mask 100 is an image in which the pixel values in the common low-entropy regiones are set to one while the value of all other pixels is set to zero. The similarity computing unit 2040 may perform a pixel-wise logical AND operation on the low-entropy region mask 80 and the low-entropy region mask 90 to generate the common low-entropy region mask 100.

<Computation of Degree of Similarity: S112>
　　The similarity computing unit 2040 computes a degree of similarity between the common low-entropy regions in the first captured image 10 and the common low-entropy regions in the shifted second captured image 20 (S112). For example, the similarity computing unit 2040 performs pixel-wise multiplication between the first captured image 10 and the common low-entropy region mask to remove the pixels outside the common low-entropy regions from the first captured image 10, thereby generating a first masked image. Similarly, the similarity computing unit 2040 performs pixel-wise multiplication between the shifted image 70 and the common low-entropy region mask to remove the pixels outside the common low-entropy regions from the shifted image 70, thereby generating a second masked image.

　　Then, the similarity computing unit 2040 computes a degree of similarity between the first masked image and the second masked image, thereby computing the degree of similarity between the pixel values of the common low-entropy regions in the first captured image 10 and the pixel values of the common low-entropy regions in the shifted image 70.

　　There are various indices that can be used to represent a degree of similarity between two images. In some implementations, mutual information is used as the index. In this case, the similarity computing unit 2040 computes mutual information between the first masked image and the second masked image. This mutual information represents the mutual information between the pixel values of the common low-entropy regions in the first captured image 10 and the pixel values of the common low-entropy regions in the shifted image 70.

<Determination of Coregistration: S116>
　　The determining unit 2060 determines the amount of shift with which the first captured image 10 and the second captured image 20 are coregistered each other (S116). Conceptually, the more similar the common low-entropy regions in the first captured image 10 and the common low-entropy regions in the shifted image 70 are, the better the second captured image 20 is aligned with the first captured image 10.

　　Thus, for example, the determining unit 2060 determines the maximum degree of similarity among all the degrees of similarity computed by the similarity computing unit 2040. Then, the determining unit 2060 determines the amount of shift corresponding to the maximum degree of similarity as the amount of shift with which the first captured image 10 and the second captured image 20 are coregistered each other.

<Output of Result>
　　The coregistering apparatus 2000 may output information, referred to as "output information," that shows the result of the coregistration of the first captured image 10 and the second captured image 20. For example, the output information may include the amount of shift with which the first captured image 10 and the second captured image 20 are coregistered each other. The output information may also include the shifted image 70 that matches the first captured image 10 (in other words, the second captured image 20 that is best aligned with the first captured image 10).

　　The output information may be output in various manners. For example, the coregistering apparatus 2000 puts the output information into a storage unit. In another example, the coregistering apparatus 2000 outputs the output information to a display device so as to be displayed by the display device. In another example, the coregistering apparatus 2000 sends the output information to another apparatus, such as one that is used by a user of the coregistering apparatus 2000.

　　The program can be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g., magneto-optical disks), CD-ROM (compact disc read only memory), CD-R (compact disc recordable), CD-R/W (compact disc rewritable), and semiconductor memories (such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, RAM (random access memory), etc.). The program may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g., electric wires, and optical fibers) or a wireless communication line.

　　Although the present disclosure is explained above with reference to example embodiments, the present disclosure is not limited to the above-described example embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the invention.

　　The whole or part of the example embodiments disclosed above can be described as, but not limited to, the following supplementary notes.
<Supplementary notes>
(Supplementary Note 1)
　　A coregistering apparatus comprising:
　　at least one memory that is configured to store instructions; and
　　at least one processor that is configured to execute the instructions to:
　　acquire a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　perform, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determine, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 2)
　　The coregistering apparatus according to supplementary note 1,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
(Supplementary Note 3)
　　The coregistering apparatus according to supplementary note 1,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
(Supplementary Note 4)
　　The coregistering apparatus according to supplementary note 1,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 5)
　　The coregistering apparatus according to supplementary note 1,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.
(Supplementary Note 6)
　　A coregistering method, performed by a computer, comprising:
　　acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　performing, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 7)
　　The coregistering method according to supplementary note 6,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
(Supplementary Note 8)
　　The coregistering method according to supplementary note 6,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
(Supplementary Note 9)
　　The coregistering method according to supplementary note 6,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 10)
　　The coregistering method according to supplementary note 6,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.
(Supplementary Note 11)
　　A non-transitory computer-readable medium storing a program that causes a computer to execute:
　　acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　performing, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 12)
　　The non-transitory computer-readable medium according to supplementary note 11,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
(Supplementary Note 13)
　　The non-transitory computer-readable medium according to supplementary note 11,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
(Supplementary Note 14)
　　The non-transitory computer-readable medium according to supplementary note 11,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
(Supplementary Note 15)
　　The non-transitory computer-readable medium according to supplementary note 11,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.

10　　first captured image
20　　second captured image
30　　target place
50　　sensor
60　　sensor
70　　shifted image
80　　low-entropy region mask
90　　low-entropy region mask
100　　common low-entropy region mask
1000　　computer
1020　　bus
1040　　processor
1060　　memory
1080　　storage device
1100　　input/output interface
1120　　network interface
2000　　coregistering apparatus
2020　　acquiring unit
2040　　similarity computing unit
2060　　determining unit

Claims

　　A coregistering apparatus comprising:
　　at least one memory that is configured to store instructions; and
　　at least one processor that is configured to execute the instructions to:
　　acquire a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　perform, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determine, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
　　The coregistering apparatus according to claim 1,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
　　The coregistering apparatus according to claim 1,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
　　The coregistering apparatus according to claim 1,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
　　The coregistering apparatus according to claim 1,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.
　　A coregistering method, performed by a computer, comprising:
　　acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　performing, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
　　The coregistering method according to claim 6,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
　　The coregistering method according to claim 6,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
　　The coregistering method according to claim 6,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
　　The coregistering method according to claim 6,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.
　　A non-transitory computer-readable medium storing a program that causes a computer to execute:
　　acquiring a first captured image and a second captured image, the first captured image and the second captured image being generated by capturing a same place as each other from different angles from each other;
　　performing, for each one of two or more different amounts of shift to be applied to the second captured image:
　　　　shifting the second captured image;
　　　　detecting one or more common low-entropy regions, which are regions having lower entropy than a threshold in both of the first captured image and the shifted second captured image; and
　　　　computing a degree of similarity between the one or more common low-entropy regions in the first captured image and the one or more common low-entropy regions in the shifted second captured image; and
　　determining, based on the degree of similarity computed for each amount of shift applied to the second captured image, the amount of shift with which the first captured image and the second captured image are coregistered.
　　The non-transitory computer-readable medium according to claim 11,
　　wherein the detection of the common low-entropy regions includes:
　　　　detecting one or more low-entropy regions from the first captured image, the low-entropy region being a region having lower entropy than the threshold;
　　　　detecting one or more low-entropy regions from the second captured image; and
　　　　detecting, as the common low-entropy regions, one or more regions that are common in the low-entropy regions detected from the first captured image and the low-entropy regions detected from the second captured image.
　　The non-transitory computer-readable medium according to claim 11,
　　wherein mutual information between pixel values of the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image is computed as the degree of similarity between the one or more common low-entropy regions in the first captured image and pixel values of the one or more common low-entropy regions in the shifted second captured image.
　　The non-transitory computer-readable medium according to claim 11,
　　wherein the determination of the amount of shift with which the first captured image and the second captured image are coregistered includes:
　　　　determining a maximum of the computed degrees of similarity; and
　　　　determining the amount of shift corresponding to the determined maximum as the amount of shift with which the first captured image and the second captured image are coregistered.
　　The non-transitory computer-readable medium according to claim 11,
　　wherein the first captured image and the second captured image are synthetic-aperture radar images.