WO2018198212A1

WO2018198212A1 - Information processing device, information processing method, and computer-readable storage medium

Info

Publication number: WO2018198212A1
Application number: PCT/JP2017/016451
Authority: WO
Inventors: 健太先崎
Original assignee: 日本電気株式会社
Priority date: 2017-04-26
Filing date: 2017-04-26
Publication date: 2018-11-01

Abstract

The present invention facilitates the understanding of a point on an observed object that contributes to the signal at a point in a region in which there is layover in an intensity map of a signal from the observed object acquired through radar. An information processing device (11) according to an embodiment is provided with: a candidate point extraction unit (114) for extracting, on the basis of the position in three-dimensional space of a target point specified in an intensity map (1111) of a signal from an observed object acquired through radar and of the shape of the observed object, a candidate point that contributes to the signal at the target point; a display mode determination unit (115) for determining, on the basis of the position in three-dimensional space of the target point and the photography conditions for a spatial image (1114) showing the observed object, a display mode for display indicating the position of the candidate point in the spatial image (1114); and an image generation unit (116) for generating an image indicating the position of the candidate point in the spatial image (1114) using the determined display mode.

Description

Information processing apparatus, information processing method, and computer-readable storage medium

This disclosure relates to processing of data acquired by a radar.

For the purpose of observing the state of the earth's surface, etc., a technique for observing and analyzing the area to be observed from the sky is widespread.

Synthetic Aperture Radar (SAR) is a technology that observes the state of the earth's surface by irradiating electromagnetic waves from above and acquiring the intensity of the reflected electromagnetic waves (hereinafter also referred to as “reflected waves”). One. Based on the data acquired by the SAR, a two-dimensional map (hereinafter, “SAR image”) of the intensity of the reflected wave can be generated. The SAR image is a map in which the reflected wave is regarded as a reflected wave from a defined reference plane (for example, the ground surface) and the intensity of the reflected wave is represented on a plane representing the reference plane.

The position where the intensity of the reflected wave is represented in the SAR image is based on the distance between the position where the reflected wave is generated and the position of the antenna that receives the reflected wave. Therefore, the intensity of the reflected wave from a position away from the reference plane is represented in the SAR image at a position shifted to the radar side according to the height from the reference plane with respect to the actual position. As a result, the image formed in the SAR image by the reflected wave from the object whose shape is not flat is an image in which the shape of the actual object is distorted. The phenomenon in which such a distorted image is generated is called foreshortening.

Patent Documents 1 and 2 disclose apparatuses that perform correction processing called ortho correction in order to correct foreshortening.

Patent Document 3 discloses a technique for correcting not only foreshortening but also a phenomenon called layover. The layover is a phenomenon in which a reflected wave signal from a position at a certain height and a reflected wave signal from a position different from the position overlap in the SAR image.

Note that Patent Document 4 which is a document related to the present disclosure has a description regarding an occlusion area in an image photographed by a camera.

JP 2007-248216 A JP 2008-90808 A JP 2008-185375 A JP 2014-160405 A

The ortho correction as disclosed in Patent Documents 1 and 2 is not assumed to be performed on a SAR image in which a layover has occurred. Specifically, the ortho correction is a correction in which the position of a point where distortion occurs in the SAR image is shifted to a position estimated as a true position where a signal (reflected wave) represented at the point is generated. In other words, the ortho correction is a correction that is performed on the assumption that there is one position candidate that is estimated as the true position where the reflected wave is emitted at the point to be corrected.

In the ortho correction as disclosed in Patent Documents 1 and 2, it is not possible to correct a point in a region where a layover occurs. This is because when there is a layover, there may be a plurality of position candidates estimated as the true position from which a signal represented at a point existing in the area where the layover occurs.

Patent Document 3 discloses a method for correcting layover, but this method requires a plurality of SAR images having different distortion methods. Thus, in the absence of any supplemental information, the reflected waves from two or more points that contribute to the signal at a point in the region where layover occurs in one SAR image are distinguished. That is impossible in principle.

If the layover is not corrected, that is, if a candidate point that contributes to a signal at a certain point in the SAR image is not narrowed down, a person can select a candidate point that contributes to the signal while viewing the SAR image and the optical image. It is customary to estimate based on experience and various information.

However, it is difficult to understand a SAR image and estimate a candidate point that contributes to a signal indicated by a point in the SAR image.

It is an object of the present invention to provide an apparatus, a method, and the like that make it easy to understand a point that contributes to a signal at a point in a region where a layover occurs in a SAR image. However, the images used in the present invention are obtained by other methods for estimating the state of an object by observing the reflection of electromagnetic waves such as images based on RAR (Real Aperture Radar) in addition to SAR images. It may be an image.

An information processing apparatus according to an aspect of the present invention includes a position in a three-dimensional space of a target point, which is a point specified in an intensity map of a signal from an observed object acquired by a radar, and a shape of the observed object And candidate display means for extracting candidate points that are points that contribute to the signal at the target point, and a display mode for displaying the position of the candidate point in the spatial image in which the observed object is captured Display mode determining means for determining the position of the candidate point based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image, and the position of the candidate point in the spatial image is displayed according to the determined display mode. Image generating means for generating a processed image.

An information processing method according to an aspect of the present invention includes a position in a three-dimensional space of a target point, which is a point specified in an intensity map of a signal from an observed object acquired by a radar, and a shape of the observed object Based on the above, a candidate point that is a point contributing to the signal at the target point is extracted, and a display mode of display indicating the position of the candidate point in the spatial image in which the observed object is captured The position is determined based on the position in the three-dimensional space and the shooting condition of the space image, and an image is generated in which the position of the candidate point in the space image is displayed according to the determined display mode.

A program according to an aspect of the present invention provides a computer with a position in a three-dimensional space of a target point, which is a point specified in an intensity map of a signal from an observed object acquired by a radar, and the observed object. A candidate point extracting means for extracting a candidate point that is a point contributing to the signal at the target point based on a shape, and a display mode for displaying a position of the candidate point in a spatial image in which the object is observed Display mode determining means for determining the position of the candidate point based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image, and the display mode of determining the position of the candidate point in the spatial image And an image generation process for generating the displayed image. The program is stored in, for example, a computer-readable non-volatile storage medium.

According to the present invention, in the intensity map of the signal from the observed object acquired by the radar, it is easy to understand the point on the observed object that contributes to the signal at the point in the region where the layover occurs. can do.

It is a figure for demonstrating a layover. It is an example of a SAR image. It is a block diagram which shows the structure of the information processing apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the example of a candidate point. It is a figure for demonstrating another example of the calculation method of a candidate point. It is a figure for demonstrating an occlusion area | region. It is a figure for demonstrating the method to determine whether a candidate point is located in an occlusion area | region. It is a figure which shows the example of the property regarding a display mode, and its value. It is a flowchart which shows the flow of the process regarding extraction of a candidate point by the information processing apparatus which concerns on 1st Embodiment. It is a flowchart which shows the flow of the process regarding the production | generation of the space image by which the candidate point was shown by the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows one example of the spatial image in which the candidate point was shown. It is a figure which shows one example of the spatial image in which the candidate point was shown. It is a figure which shows one example of the spatial image in which the candidate point was shown. It is a figure which illustrates the spatial image in which the candidate point was shown when a display mode determination part did not function. It is an example of the image produced | generated by the image generation part in the modification 2 of 1st Embodiment. It is a figure which shows the example of the figure which shows a three-dimensional direction. It is a block diagram which shows the structure of the information processing apparatus which concerns on the modification 3 of 1st Embodiment. It is an example of the image produced | generated by the image generation part which concerns on the modification 3. It is a block diagram which shows the structure of the information processing apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the flow of operation | movement of the information processing apparatus which concerns on one Embodiment. It is a block diagram which shows the example of the hardware which comprises each part of each embodiment of this invention.

Prior to the description of the embodiment of the present invention, the principle of the occurrence of layover in observation by SAR will be described.

FIG. 1 is a diagram for explaining layover. In FIG. 1, an observation device S ₀ that performs observation by SAR and an object M that exists in the observed range are shown. Observation device S _0, for example, mounting the radar, a satellite or aircraft or the like. Observation device S _0, while moving the sky, transmits electromagnetic waves by the radar receives the reflected waves. In FIG. 1, the arrows indicate the traveling direction of the observation device S ₀ , that is, the radar traveling direction (also referred to as azimuth direction). Observation electromagnetic waves emitted from the device S ₀ is the surface, and reflected by the back-scattered by the structure M in the ground, part of the reflected wave is received back to the radar. Thereby, the distance between the electromagnetic wave reflection point of the position and the structure M of the observation devices S ₀ is specified.

In FIG. 1, a point Q _a is a point on the ground surface, and a point Q _b is a point away from the ground surface on the surface of the structure M. The distance between the observation equipment _{S 0} and the point _{Q a} is equal to the distance between the observation equipment _{S 0} and the point _{Q b.} Further, the straight line connecting the point Q _b and the point Q _a, and the traveling direction of the radar, are in a vertical relationship. In such a case, the reflected wave at the point Q _a, the reflected wave at the point Q _b, can not be distinguished by taking the observation equipment S _0. That is, the intensity of the reflected waves from the intensity of the reflected wave and the point Q _b from the point Q _a, is observed intermingled.

An example of an image representing the intensity distribution of the reflected wave (hereinafter referred to as “SAR image”) generated in such a case is shown in FIG. In FIG. 2, the arrow indicates the traveling direction of the radar. The SAR image is generated based on the intensity of the reflected wave received by the radar and the distance between the point where the reflected wave is emitted and the radar. In the SAR, reflected waves from two or more points that are on the plane perpendicular to the traveling direction of the radar including the position of the radar and are equal in distance from the radar are not distinguished. Point P is a point which reflects the intensity of the reflected wave from a point Q _a, the strength indicated in this respect P, is also reflected in the strength of the reflected wave from the point Q _b. In this way, a phenomenon in which the intensity of reflected waves from two or more points overlaps at one point in the SAR image is layover. In FIG. 2, a white area including the point P is an area where a layover has occurred. In FIG. 2, an area painted black represents an area shaded by the structure M against the radar. This region is also called radar shadow.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<< First Embodiment >>
First, a first embodiment of the present invention will be described.

<Configuration>
In the following description, it is assumed that a reference three-dimensional space is defined in the processing performed by the information processing apparatus 11. A three-dimensional coordinate system is defined for the reference three-dimensional space. Hereinafter, this three-dimensional coordinate system is referred to as a reference three-dimensional coordinate system or a reference coordinate system. The reference coordinate system may be, for example, a geodetic system or a coordinate system of model data 1113 that is three-dimensional data described later.

Further, hereinafter, the fact that the points described under the first coordinate system can be described under the second coordinate system, the first coordinate system is related to the second coordinate system. Is written.

FIG. 3 is a block diagram showing a configuration of the information processing apparatus 11 according to the first embodiment. The information processing apparatus 11 includes a storage unit 111, a feature point extraction unit 112, a geocoding unit 113, a candidate point extraction unit 114, a display mode determination unit 115, an image generation unit 116, and a display control unit 117. The storage unit 111, the feature point extraction unit 112, the geocoding unit 113, the candidate point extraction unit 114, the display mode determination unit 115, and the image generation unit 116 are connected so as to be able to transmit / receive data to / from each other. Note that data transmission between each unit of the information processing apparatus 11 may be performed directly via a signal line, or may be performed by reading and writing to a shared storage area (for example, the storage unit 111). In the following description, data movement is described by the words “send data” and “receive data”, but the method of transmitting data is not limited to the method of transmitting directly.

The information processing apparatus 11 is connected to the display device 21 so as to be communicable.

=== Storage 111 ===
The storage unit 111 stores data necessary for processing by the information processing apparatus 11. For example, the storage unit 111 stores SAR data 1111, SAR data parameters 1112, model data 1113, an aerial image 1114, and shooting condition information 1115.

SAR data 1111 is data obtained by observation using SAR. Targets observed by the SAR (hereinafter also referred to as “observed object”) are, for example, the ground surface and buildings. The SAR data 1111 is data that can generate at least a SAR image represented under a coordinate system related to a reference coordinate system. For example, the SAR data 1111 includes an observation value and information associated with the observation value. The observed value is, for example, the intensity of the observed reflected wave. The information associated with the observation value is, for example, the position and traveling direction of the radar that observed the reflected wave, and the position between the reflected point and the radar derived from the observation of the reflected wave. Information such as distance. The SAR data 1111 may include information on the depression angle of the radar (the elevation angle of the radar viewed from the reflection point) with respect to the object to be observed. The information regarding the position is described by, for example, a combination of longitude, latitude, and altitude in the geodetic system.

The SAR data 1111 may be a SAR image itself.

In the description of the present embodiment, observation data by SAR is assumed as data to be used. However, in other embodiments, data of observation results by, for example, RAR (Real Aperture Radar) is used instead of SAR. May be used.

The SAR data parameter 1112 is a parameter indicating the relationship between the data included in the SAR data 1111 and the reference coordinate system. In other words, the SAR data parameter 1112 is a parameter for assigning a position in the reference coordinate system to the observation value included in the SAR data 1111.

For example, when information regarding the position and direction of the radar and the distance between the radar and the observed object described under the geodetic system is associated with the observation value in the SAR data 1111, the SAR data parameter Reference numeral 1112 denotes a parameter for converting the information into information described under a reference coordinate system.

When the SAR data 1111 is a SAR image, the coordinate system of the SAR image is related to the reference coordinate system by the SAR data parameter 1112. That is, an arbitrary point in the SAR image is associated with one point in the reference coordinate system.

The model data 1113 is data representing the shape of an object in three dimensions, such as topography and building structure. The model data 1113 is, for example, DEM (Digital Elevation Model; digital elevation model). The model data 1113 may be DSM (Digital Surface Model) that is data of the earth surface including the structure, or DTM (Digital Terrain Model) that is data of the shape of the ground surface. The model data 1113 may have DTM and three-dimensional data of a structure separately.

The coordinate system used for the model data 1113 is related to the reference coordinate system. That is, an arbitrary point in the model data 1113 can be described by coordinates in the reference coordinate system.

The space image 1114 is an image in which a space including the SAR observed object is copied. The spatial image 1114 may be, for example, any of optical images such as satellite photographs and aerial photographs, maps, topographic maps, and CG (Computer Graphics) images representing the topography. The aerial image 1114 may be a projection view of the model data 1113. Preferably, in the spatial image 1114, the geographical shape and arrangement of the object in the space represented by the spatial image 1114 are viewed by the user of the information processing apparatus 11 (that is, the image output by the information processing apparatus 11). It is an image that is easy to understand intuitively.

The spatial image 1114 may be captured from outside the information processing apparatus 11 or may be generated by projecting the model data 1113 by the image generation unit 116 described later.

Shooting condition information 1115 is information related to shooting conditions (capturing conditions) of the spatial image 1114. The imaging condition of the spatial image 1114 is how the spatial image 1114 is acquired. The shooting condition information 1115 is information that can uniquely specify the shooting range of the spatial image 1114. The shooting condition information 1115 is represented by a plurality of parameter values related to the shooting range of the spatial image 1114, for example.

In the present disclosure, it is assumed that the spatial image is a captured image captured from a specific position, and a subject (for example, an imaging device such as a camera) that has performed the imaging is referred to as an imaging object. In the case where the spatial image 1114 is an image obtained without actually performing the photographing process by the apparatus, such as when the spatial image 1114 is generated by projecting the model data 1113, the photographing body may be virtually assumed. .

The photographing condition information 1115 is described by, for example, the position of the photographing object and information indicating the range of the photographing object. As an example, when the spatial image 1114 is rectangular, the imaging condition information 1115 includes four coordinates in the reference coordinate system corresponding to the coordinates in the reference coordinate system of the photographic object and the points that are captured at the four corners of the spatial image 1114. And may be described by coordinates. In this case, the shooting range is an area surrounded by four half lines extending from the position of the shooting body to the four coordinates.

Note that the position of the photographing object is strictly the position of the viewpoint of the photographing object with respect to the spatial image 1114, but in practice, the information on the position of the photographing object used by the display mode determination unit 115 does not have to be exact. . As an example, the display mode determination unit 115 is acquired by a device having a GPS (Global Positioning System) function mounted on a device (aircraft, artificial satellite, etc.) on which the photographing body is mounted as information indicating the position of the photographing body. Position information may be used.

Note that the information indicating the position in the shooting condition information 1115 is given by, for example, a set of values of parameters (for example, longitude, latitude, and altitude) in the reference coordinate system. In other words, the position in the reference three-dimensional space of any point included in the range of the space included in the spatial image 1114 can be uniquely specified by the shooting condition information 1115. On the other hand, when an arbitrary point (at least a feature point and a candidate point described later) in the reference three-dimensional space is included in the spatial image 1114, the spatial image of the point is based on the shooting condition information 1115. The position at 1114 may be uniquely identified.

Each parameter of the imaging condition information 1115 may be a parameter of a coordinate system different from the reference coordinate system. In that case, the imaging condition information 1115 only needs to include a conversion parameter for converting the parameter value in the coordinate system to the parameter value in the reference coordinate system.

The photographing condition information 1115 may be described by, for example, the position, posture, and angle of view of the photographing body. The posture of the photographing object can be described by a photographing direction, that is, an optical axis direction of the photographing object at the time of photographing, and a parameter indicating a relationship between a vertical direction of the spatial image 1114 and a reference coordinate system. For example, when the spatial image 1114 is rectangular, the angle of view can be described by parameters indicating a vertical viewing angle and a horizontal viewing angle.

When the photographic body is sufficiently far from the subject, such as when the photographic body is a camera mounted on an artificial satellite, the information indicating the position of the photographic body is described by the value of a parameter indicating the direction of the photographic body viewed from the subject. May be. For example, the information indicating the position of the photographic object may be a set of azimuth and elevation angle.

The shooting condition information 1115 may include shooting time information.

Note that the storage unit 111 does not always need to hold data in the information processing apparatus 11. For example, the storage unit 111 may record data on a device or a recording medium outside the information processing apparatus 11 and acquire the data as necessary. That is, the storage unit 111 only needs to be configured to acquire data requested by each unit in the processing of each unit of the information processing apparatus 11 described below.

=== Feature Point Extractor 112 ===
The feature point extraction unit 112 extracts feature points from the SAR data 1111. In the present disclosure, a feature point is a point extracted by a predetermined method from points indicating signal intensity that is not at least 0 in the SAR data 1111. That is, the feature point extraction unit 112 extracts one or more points from the SAR data 1111 by a predetermined method for extracting points. In the present disclosure, the points extracted from the SAR data 1111 are a data group related to one point in the SAR image (for example, a set of an observation value and information associated with the observation value).

The feature point extraction unit 112 extracts feature points by, for example, a method of extracting points that may give useful information in the analysis of the SAR data 1111.

For example, the feature point extraction unit 112 may extract points by a technique called PS-InSAR (Permanent Scatterers Interferometric SAR). PS-InSAR is a technique for extracting a point where a change in signal intensity is observed based on a phase shift from a plurality of SAR images.

Alternatively, the feature point extraction unit 112 may extract a point that satisfies a predetermined condition (for example, the signal intensity exceeds a predetermined threshold) as the feature point. This predetermined condition may be set by a user or a designer of the information processing apparatus 11, for example. The feature point extraction unit 112 may extract points selected by human judgment as feature points.

The feature point extraction unit 112 sends the extracted feature point information to the geocoding unit 113. The feature point information includes at least information capable of specifying coordinates in the reference coordinate system. For example, the feature point information is represented by the position and traveling direction of the observation device that has acquired the SAR data in the range including the feature point, and the distance between the observation device and the signal reflection point at the feature point.

=== Geocoding unit 113 ===
The geocoding unit 113 assigns coordinates in the reference coordinate system to each of the feature points extracted by the feature point extraction unit 112. For example, the geocoding unit 113 receives information on the extracted feature points from the feature point extraction unit 112. Based on the received feature point information and the SAR data parameter 1112, the geocoding unit 113 identifies from which position in the reference three-dimensional space the signal of the feature point corresponds.

For example, when the feature point information is represented by the position of the observation device that has acquired the SAR data in the range including the feature point, the traveling direction, and the distance between the observation device and the reflection point of the signal at the feature point, First, the geocoding unit 113 converts the information based on the SAR data parameter 1112 into information represented by the position, traveling direction, and distance of the observation device in the reference coordinate system. Then, the geocoding unit 113 identifies points (coordinates) that satisfy all of the following conditions in the reference coordinate system.
-The distance between the point and the position of the observation device is the distance indicated by the feature point information.
・ Included in a plane perpendicular to the direction of travel of the observation equipment.
Included in a reference plane (a plane with an altitude of 0 in the reference coordinate system).
The coordinates of the identified point are the coordinates in the reference coordinate system of the feature point indicated by the feature point information. For example, the geocoding unit 113 assigns the coordinates of the points specified in this way to the feature points indicated by the feature point information.

=== Candidate Point Extractor 114 ===
Candidate point extraction section 114 associates a point related to the feature point (hereinafter, “candidate point”) with the feature point given the coordinate in the reference coordinate system. The candidate points related to the feature points will be described below.

The signal intensity indicated at the feature point (referred to as point P) in the region where the layover occurs may be the sum of the intensity of the reflected waves from a plurality of points. At this time, a point in the three-dimensional space that may contribute to the signal intensity indicated at the point P is referred to as a candidate point related to the point P in this embodiment.

FIG. 4 is a diagram for explaining an example of candidate points. FIG. 4 is a cross-sectional view of the reference three-dimensional space cut out by a plane passing through the point P and perpendicular to the radar traveling direction (azimuth direction).

A line GL is a cross-sectional line of a reference plane in a reference three-dimensional space, that is, a plane on which a feature point is located. A line ML is a cross-sectional line of the three-dimensional structure represented by the model data 1113. Point S ₁ is a point indicating the position of the radar. The position of the point P is a coordinate position given by the geocoding unit 113. The distance between the points P and S ₁ is assumed to be "R".

Reflected in the signal intensity indicated at the point P is a reflected wave from a point whose distance from the point S ₁ is “R” in the cross-sectional view. That is, the point involved in the point P is the point at which arc centered on the point S ₁ radius "R" intersects the line ML. In FIG. 4, points Q ₁ , Q ₂ , Q ₃ , and Q ₄ are points other than the point P where the circular arc with the radius “R” centering on the point S ₁ intersects the line ML. Therefore, these points Q ₁ , Q ₂ , Q ₃ , Q ₄ are candidate points related to the point P.

In this way, the candidate point extraction unit 114 extracts, as candidate points, points on the plane that includes the point P and that is perpendicular to the traveling direction of the radar and whose distance from the radar is equal to the distance between the radar and the point P. That's fine.

However, since the point Q ₃ is shaded with respect to the point S ₁ (in the so-called radar shadow), there is a possibility that the electromagnetic wave reflected at this point has contributed to the signal intensity indicated at the point P. Low. Therefore, the candidate points extracted by the candidate point extraction unit 114 may be points Q ₁ , Q ₂ , Q ₄ excluding the point Q ₃ . In other words, the candidate point extracting unit 114, based on the line segment connecting the point Q ₃ and the point S ₁ is crossing the line ML outside point Q _3, may exclude the point Q ₃ from the candidate point.

Information required in the extraction of candidate points as described above, the three-dimensional space as a reference, according to a plane perpendicular to the point P as the azimuth direction, the section line of the model data 1113, the position of the point S ₁ and the point P, In addition, the distance “R” between the point S ₁ and the point P.

When the point S ₁ is sufficiently far away, it can be approximated that the incident directions of electromagnetic waves from the point S ₁ to the object to be observed are all parallel to each other. Therefore, when the point S ₁ is sufficiently distant, as shown in FIG. 5, the candidate point, passes through the point P, and a line perpendicular to the electromagnetic wave of the incident beam to the point P from the radar, the intersection of the line ML It can be specified by seeking. However, in FIG. 5, a candidate for point Q ₃ is that a straight line passing through point Q ₃ and parallel to the incident line of the electromagnetic wave from the radar intersects line ML (ie, is in the radar shadow). May be removed from the point. In this way, the candidate point extraction unit 114 may extract candidate points under the approximation that the incident directions of the electromagnetic waves from the observation device to the object to be observed are all parallel to each other. In the extraction by such a method, the position of the candidate point can be calculated using the azimuth and depression angle θ of the point S ₁ instead of the coordinate of the point S ₁ and the distance “R”.

The candidate point extraction unit 114 sends candidate points related to the feature points to the display mode determination unit 115 and the image generation unit 116.

=== Display Mode Determination Unit 115 ===
The display mode determination unit 115 determines the display mode of the candidate points displayed in the image generated by the image generation unit 116, which will be described later, based on the position of the candidate points in the three-dimensional space and the shooting condition information 1115. .

The display mode is a state of display determined by, for example, the shape, size, color, brightness, transparency, movement, and change with time of a figure to be displayed. In the present disclosure, the “candidate point display mode” is a display mode for displaying the position of the candidate point. “Display candidate points” means to display a display indicating the positions of candidate points.

The shooting condition is indicated by the shooting condition information 1115 as described above.

The display mode determination unit 115 receives the coordinates of the candidate points from the candidate point extraction unit 114 when determining the display mode of the candidate points. Further, the display mode determination unit 115 reads out the model data 1113 and the imaging condition information 1115 of the spatial image used for the image generated by the image generation unit 116 from the storage unit 111.

Hereinafter, a specific example of a method in which the display mode determination unit 115 determines the display mode will be described.

(1) Determination of display mode in consideration of blind spot The display mode determination unit 115 displays different display modes depending on whether the candidate point is positioned in a region that is a blind spot in a spatial image in a three-dimensional space, for example. decide.

An area that is a blind spot in a spatial image is an area that is included in the imaging range of the spatial image, but is blocked by an object that appears in the spatial image and is not visible from the position of the photographic object that captured the spatial image. . In the present embodiment, an area that becomes a blind spot in a spatial image is also referred to as an occlusion area.

FIG. 6 is a diagram for explaining the occlusion area. 6, the point S ₂ represents the position of the imaging member. The solid M is a rectangular parallelepiped structure. The range of the three-dimensional area indicated by the dotted line is the shooting range. Straight Lc, Ld, Le, respectively, through a point Qc on sterically M, Qd, the Qe, is a straight line extending from the position of the point _{S 2.} A three-dimensional area indicated by diagonal lines in FIG. 6 is an occlusion area.

For example, the display mode determination unit 115 first determines whether each candidate point is located in the occlusion area.

As an example, the display mode decision unit 115, the point Q ₂ will be described a method of determining whether located in the occlusion region. FIG. 7 is a diagram for explaining a method for determining whether a candidate point is located in the occlusion area. Figure 7 is a S ₂ points representing the position of the imaging of the spatial image, which is a diagram of a plane including the point Q _2. A line GL is a cross-sectional line of the reference plane of the reference three-dimensional space, and a line ML is a cross-sectional line of the three-dimensional structure represented by the model data 1113.

Display mode decision unit 115 first calculates a line segment connecting the candidate points _{Q 2} and the point _{S 2.} The display mode decision unit 115 determines that whether the segment has an intersection outside line ML and the point Q ₂ (i.e., intersects the model data 1113). When the line segment has an intersection with the line ML, the display mode determination unit 115 determines that the candidate point is located in the occlusion area. When the line segment does not have an intersection with the line ML, the display mode determination unit 115 determines that the candidate point is not located in the occlusion area. In the example of FIG. 7, a line segment connecting the point _{Q 2} and the point _{S 2} has a line ML and the intersection _{Q f.} Thus, the display mode determination unit 115 determines that the candidate point Q ₂ is positioned in the occlusion region.

The display mode determination unit 115 determines whether or not the candidate points included in the imaging range of the spatial image are located in the occlusion area, for example, as described above.

Incidentally, in the above description, the line segment connecting the candidate points and the point S ₂ may be a half line extending in the direction of the imaging member from the candidate point. When the position of the point S ₂ is sufficiently far from the observed object, the direction of the imaging member to each of the candidate points are considered to be identical. That is, the display mode determination unit 115, by the point S ₂ is regarded as sufficiently far, all of the candidate points in space image, the determination can be performed by using a half-line in the same direction. In this case, the calculation cost related to the determination can be reduced.

After the determination, the display mode determination unit 115 determines the display mode of each candidate point so that the display mode of candidate points located in the occlusion area is different from the display mode of candidate points not positioned in the occlusion area.

The display mode determination unit 115 associates information indicating the determined display mode with candidate points. For example, the display mode determination unit 115 may set a property value related to the display mode of each candidate point. Or the display mode determination part 115 should just associate the identification information of the set of the property value relevant to the display mode already prepared with a candidate point. To associate the second information with the first information is to generate data indicating that the first information and the second information are associated with each other.

For example, the display mode determination unit 115 sets the transmittance value of the candidate points not located in the occlusion area to a value different from the transmittance of the candidate points located in the occlusion area. The transmittance is a parameter indicating the degree of contribution to the pixel value at the superimposed position when the displayed graphic is superimposed on the image. For example, when a graphic with a transmittance of 0% is superimposed, the pixel value at the superimposed position of the graphic depends only on the color of the graphic. When a graphic with a transmittance of 0% is superimposed, the pixel value at the superimposed position of the graphic is a value that also depends on the color of the pixel value before superimposition. In other words, a graphic whose transmittance is not 0% is displayed as a semitransparent graphic.

FIG. 8 is a display mode showing examples of properties relating to the display mode and examples of values of each property in two types of display modes (“first display mode” and “second display mode”). The display mode determination unit 115 stores, for example, data representing the contents of the table as shown in FIG. 8 in advance, or stores the data in the storage unit 111, for each candidate point. One display mode "or" second display mode "may be associated. Note that as a property related to the display mode, there may be a property for designating whether or not a graphic is displayed.

When the table shown in FIG. 8 is used, for example, the display mode determination unit 115 associates the first display mode as a display mode of candidate points that are not located in the occlusion area, and sets the first display mode of candidate points that are positioned in the occlusion area. The two display modes may be associated with each other. By doing so, the candidate points located in the occlusion area are displayed in the transmissive color on the display device 21.

Note that there may be a display mode in which the value of the “transmission” property is “100%”. Further, the transmittance may be set for each portion of the graphic to be displayed (such as an outline and an internal region).

(1 ′) Determination of the display mode in consideration of the stage of the blind spot In the example described above, the display mode determination unit 115 displays different display modes according to the number of times the half line extending from the candidate point to the observation device intersects the line ML. You may determine with respect to each candidate point.

For example, the display mode determination unit 115 seems to have a transmittance of 50% for the candidate point where the number of times the half line intersects the line ML is one time, and the number of times that the half line intersects the line ML is two times or more. The transmittance of a candidate point may be set to 80%.

This makes it possible to deepen the understanding of the positions of candidate points located in the occlusion area. For example, the viewer can understand whether the candidate point displayed in the transparent color is located on the wall of the building or further on the ground based on the degree of transmittance.

(2) Determination of display mode according to distance The display mode determination unit 115 may determine a display mode that changes according to the distance between the candidate point and the photographic object, for example.

The display mode determination unit 115 may determine the display mode of each candidate point so that, for example, as the candidate point is farther from the photographic object, the size of the graphic indicating the candidate point becomes smaller. Or the display mode determination part 115 may determine the display mode of each candidate point so that the brightness of the figure which shows a candidate point becomes low, for example, so that a candidate point is far from a to-be-photographed body. Such a configuration makes it easy to understand the positional relationship between candidate points.

When the display mode of each candidate point is determined, the display mode determination unit 115 sends the display mode of each candidate point to the image generation unit 116.

=== Image Generation Unit 116 ===
The image generation unit 116 generates an image to be displayed on the display device 21 by the display control unit 117. Specifically, the image generation unit 116 generates an image in which a display indicating the position of the candidate point is superimposed on the spatial image. In the present disclosure, “generating an image” means generating data for displaying an image. The format of data generated by the image generation unit 116 is not limited to the image format. The image generated by the image generation unit 116 may be data having information necessary for the display device 21 to display.

The image generation unit 116 reads a spatial image used for the generated image from the spatial image 1114 stored in the storage unit 111. The image generation unit 116 may determine the image to be read based on an instruction from the user, for example. For example, the image generation unit 116 may receive information specifying one of the plurality of spatial images 1114 from the user. Alternatively, for example, the image generation unit 116 may receive information specifying a range in the three-dimensional space and read a spatial image including the specified range.

Alternatively, the image generation unit 116 may accept information designating feature points or candidate points that the user desires to display. Then, the image generation unit 116 may specify a range in the reference three-dimensional space including the designated feature point or candidate point, and read a spatial image including the specified range. Note that the information that specifies the feature points or candidate points that the user desires to display may be information that specifies the SAR data 1111.

The image generation unit 116 may cut out a part of the spatial image 1114 stored in the storage unit 111 and read it as a spatial image to be used. For example, when the image generation unit 116 reads a spatial image based on candidate points that the user desires to display, the image generation unit 116 cuts out a range including all the candidate points from the spatial image 1114 and uses the cut-out image as a spatial image. May be read out.

Also, the image generation unit 116 receives the coordinates of the candidate points extracted by the candidate point extraction unit 114 from the candidate point extraction unit 114. Further, the image generation unit 116 acquires information indicating the display mode of each candidate point from the display mode determination unit 115.

Then, the image generation unit 116 superimposes a display indicating the position of the candidate point extracted by the candidate point extraction unit 114 on the read spatial image according to the display mode determined by the display mode determination unit 115. Thereby, a spatial image in which candidate points are shown is generated.

Hereinafter, the spatial image generated by the image generation unit 116 and indicating the candidate points is also referred to as a “point display image”.

A specific example in which the image generation unit 116 specifies the position of the candidate point in the spatial image will be described.

The image generation unit 116 may specify the position of the candidate point in the spatial image by calculation based on the shooting condition information 1115.

For example, the image generation unit 116 specifies the shooting range and shooting direction of the spatial image based on the shooting condition information 1115. Then, the image generation unit 116 obtains a cut surface of the shooting range by a plane that includes the candidate point and is perpendicular to the shooting direction. The positional relationship between the cut surface and the candidate point corresponds to the positional relationship between the spatial image and the candidate point. The image generation unit 116 may specify the coordinates of the candidate points when the coordinates of the cut surface are related to the coordinates of the spatial image. The identified coordinates are the coordinates of candidate points in the spatial image.

The optical satellite image may be corrected by ortho correction or the like. When the optical satellite image is corrected, the position where the candidate point is indicated is also corrected. The position of the candidate point may be corrected using the correction parameter used in the correction for the optical satellite image.

The above-described method for specifying the position of the candidate point in the spatial image is an example. The image generation unit 116 may specify the position of the candidate point in the spatial image based on the position of the candidate point in the reference coordinate system and the relationship between the spatial image and the reference coordinate system.

The image generation unit 116 sends the generated point display image to the display control unit 117.

=== Display Control Unit 117, Display Device 21 ===
The display control unit 117 performs control to display the point display image generated by the image generation unit 116 on the display device 21. The display control unit 117 causes the display device 21 to display the point display image by, for example, outputting the point display image to the display device 21.

The display device 21 is a display such as a liquid crystal monitor or a projector. The display device 21 may have a function as an input unit like a touch panel. In the description of the present embodiment, the display device 21 is connected to the information processing device 11 as an external device of the information processing device 11, but even if the display device 21 is included in the information processing device 11 as a display unit. Good.

The viewer who sees the display on the display device 21 knows the result of the processing by the information processing device 11. Specifically, the viewer can observe the point display image generated by the image generation unit 116.

<Operation>
(Extraction of candidate points)
The flow of operations related to the process of extracting candidate points by the information processing apparatus 11 will be described with reference to the flowchart of FIG.

The feature point extraction unit 112 of the information processing apparatus 11 acquires the SAR data 1111 from the storage unit 111 (S111). The acquired SAR data 1111 includes at least SAR data in a range included in the spatial image used in step S121 described later.

Then, the feature point extraction unit 112 extracts feature points from the acquired SAR data 1111 (step S112).

Next, the geocoding unit 113 assigns coordinates indicating the position of the feature point in the reference coordinate system to the extracted feature point (step S113). The geocoding unit 113 sends the coordinates assigned to the extracted feature points to the candidate point extraction unit 114.

The candidate point extraction unit 114 extracts candidate points related to the feature point based on the coordinates of the feature point and the model data 1113 (step S114). That is, the candidate point extraction unit 114 specifies the coordinates of candidate points related to the feature points. Then, the candidate point extraction unit 114 sends the coordinates of the candidate points to the display mode determination unit 115 and the image generation unit 116. The candidate point extraction unit 114 may store the coordinates of the candidate points in the storage unit 111 in a format in which the feature points and the candidate points are associated with each other.

(Generate dot display image)
The flow of operations related to the process of generating the point display image by the information processing apparatus 11 will be described with reference to the flowchart of FIG.

The display mode determination unit 115 reads shooting condition information of a spatial image used in the process of step S124 described later (step S121). In addition, the display mode determination unit 115 acquires the coordinates of the candidate points extracted by the candidate point extraction unit 114 from the candidate point extraction unit 114. When the coordinates of the candidate points are stored in the storage unit 111, the display mode determination unit 115 may read the coordinates of the candidate points from the storage unit 111. And the display mode determination part 115 specifies the candidate point contained in the range of a spatial image based on imaging condition information (step S122).

Then, the display mode determination unit 115 determines the display mode of each candidate point included in the range of the spatial image based on the position of the candidate point included in the range of the spatial image, the model data 1113, and the shooting condition information. (Step S123). The display mode determination unit 115 sends information on the display mode of the determined candidate points to the image generation unit 116.

Then, the image generation unit 116 generates a point display image that is a spatial image indicating the position of the candidate point (step S124). Specifically, the image generation unit 116 reads a spatial image from the storage unit 111 and superimposes a display indicating the position of the candidate point on the spatial image according to the display mode determined by the display mode determination unit 115.

Note that the timing at which the spatial image used by the image generation unit 116 is determined may be before or after the timing at which the process for acquiring the SAR data is performed. That is, in one example, after the spatial image to be used is determined, the information processing apparatus 11 specifies the SAR data 1111 that is data in the range included in the determined spatial image, and the specified SAR data Alternatively, the processing from step S111 to step S114 may be executed. In one example, the information processing apparatus 11 performs the processes from step S111 to step S114 on the SAR data 1111 in a range that can be included in the spatial image 1114 in advance before the spatial image to be used is determined. May be.

The image generation unit 116 sends the generated image to the display control unit 117.

The display control unit 117 causes the display device 21 to display the point display image generated by the image generation unit 116. Thereby, the display device 21 displays the point display image generated by the image generation unit 116 (step S125). By viewing this display, the viewer can easily understand the candidate points related to the feature points extracted in the SAR data 1111.

<Effect>
According to the information processing apparatus 11 according to the first embodiment, a viewer can easily understand a point that contributes to a signal at a point in a region where a layover occurs in the SAR image. The reason is that the candidate point extraction unit 114 extracts candidate points that may have contributed to the signal at the feature point based on the model data 1113, and the image generation unit 116 displays the position of the candidate point. This is because a point display image that is a spatial image is generated.

The display mode determination unit 115 displays the position of the candidate point displayed in the point display image in a more easily understandable display mode based on the position of the candidate point and the shooting conditions.

Hereinafter, an example of a point display image in the case where the display mode determination unit 115 determines a different display mode for a candidate point depending on whether or not the candidate point is located in a region that is a blind spot in the spatial image. introduce.

FIG. 11 is a diagram illustrating an example of a dot display image displayed by the display device 21. However, the letters “Q ₅ ” and “Q ₆ ” and the curves indicating the points are shown for convenience of explanation and are not included in the actually displayed image. In the example shown in FIG. 11, a circle representing a candidate point is superimposed on an optical satellite image showing a building. In the example shown in FIG. 11, no candidate point is located in the occlusion area. Therefore, the display mode of each candidate point is uniform.

FIG. 12 is a diagram illustrating an example of a point display image in which the building and the candidate point displayed in FIG. 11 are used as an optical image taken from another position as a spatial image. In the example shown in FIG. 12, the candidate point Q ₅ and Q ₆ are to positioned in the occlusion area are displayed in a manner different from the other candidate points.

According to such a display, viewers viewing the point display image, on the surface of the candidate point Q ₅ and Q ₆ are visible building can recognize that not located. Specifically, in FIG. 12, if a surface including a white square (representing a window) of a building is defined as “front surface” for convenience of explanation, the viewer can see that candidate points Q ₅ and Q ₆ are buildings. It can be recognized that it is not located in front of.

An example of a point display image when the display mode determination unit 115 determines different display modes for the candidate points according to the distance of the candidate points from the photographic body will be introduced. FIG. 13 shows a point display with the same angle of view as the point display image shown in FIG. 12 when the display mode determination unit 115 determines the display mode so that the size of the circle indicating the candidate point differs according to the distance. It is a figure which shows an image. In the example shown in FIG. 13, circles indicate the candidate point Q ₅ and Q ₆ to be displayed smaller than the circle that indicates the other candidate points located in front of the building, the candidate point Q ₅ and Q ₆ buildings It turns out that it is not a point located in front of.

FIG. 14 is a diagram showing a point display image having the same angle of view as the point display image shown in FIG. 13 when the display mode determination unit 115 does not function. When the display mode determination unit 115 does not function, the display modes of the candidate points Q ₅ and Q ₆ are the same as the display modes of the other candidate points. In this case, the viewer will only see the point display image shown in FIG. 14, the candidate point Q ₅ and Q ₆ can not be identified whether the position to which the surface of the building. Thus, for example, viewers can even obtain useful information from the information represented by the point Q _5, the information can not determine whether the information relating to which position of a building. For example, in PS-InSAR, fluctuations in feature points can be observed using two SAR data 1111 having different data acquisition times. However, the candidate point Q ₅ is even know viewers that feature points involved varies, specifically which location can not be determined whether varied. Alternatively, the viewer may mistakenly recognize that the front of the building has changed.

Also, viewers, a signal to the degree of contribution of each candidate point is a feature point, if you are trying to determine, based on the position of the candidate point, the position of the point Q ₅ and Q ₆ are not fixed, the point Q ₅ and Q ₆ can not determine the degree contribute to the signal of the feature points.

The inconvenience as described above can be solved by the display mode determination unit 115. That is, the information processing apparatus 11 provides the viewer with more easily understandable information regarding the feature points.

<< Modification 1 >>
In the operation example of the information processing apparatus 11 described above, the order of the process of step S111 and the process of step S112 may be reversed. That is, the feature point extraction unit 112 may extract a feature point from the points given coordinates by the geocoding unit 113.

<< Modification 2 >>
The display mode determination unit 115 changes the display mode so that the display indicating the position of the candidate point further indicates the direction of the radar that received the signal from the candidate point or the incident direction of the electromagnetic wave from the radar. You may decide.

FIG. 15 is a diagram illustrating an example of a point display image when the display mode determination unit 115 determines a display mode that indicates the incident direction of the electromagnetic wave from the radar with respect to each candidate point.

In FIG. 15, the incident direction of the electromagnetic wave from the radar is indicated by an arrow that overlaps the circle indicating the candidate point. With such a display, the viewer can know the direction of the radar.

The figure showing the incident direction may be a figure showing not only a direction parallel to the image but also a three-dimensional direction. FIG. 16 is a diagram illustrating an example of a figure indicating a three-dimensional direction. The arrow shown in FIG. 16 points to the lower right direction closer to the back side of the page. For example, the incident direction is a direction indicated by an arrow. In this case, the direction opposite to the direction indicated by the arrow is the direction of the radar viewed from the candidate point. According to such a display, the viewer can more specifically know the incident direction of the electromagnetic wave from the radar.

Since the candidate point is the point where the electromagnetic wave from the radar is incident, it can be seen that the candidate point is not included in the occlusion area if at least the candidate point is viewed in the indicated direction. Therefore, the display showing the incident direction of the electromagnetic wave from the radar allows the viewer to know the shooting direction in which the candidate point is not hidden. Further, when there are a plurality of candidate positions of the displayed candidate points in the three-dimensional space, points that cannot be observed from the indicated direction can be excluded from the candidates. Therefore, the viewer may be able to deepen understanding of the position of the candidate point in the three-dimensional space.

<< Modification 3 >>
The display mode determination unit 115 may further be configured to determine the display mode so that the display mode of candidate points related to a specific feature point is different from the display mode of other candidate points.

For example, the display mode determination unit 115 may determine the display mode so that candidate points related to the feature points designated by the user are displayed in white and other candidate points are displayed in black.

The designation of feature points by the user is performed by the designation receiving unit 118, for example. FIG. 17 is a block diagram illustrating a configuration of the information processing apparatus 12 including the designation receiving unit 118.

The designation accepting unit 118 accepts designation of feature points from the user of the information processing apparatus 12, for example. For example, the information processing apparatus 12 may cause the display device 21 to display a SAR image showing the feature points. And the designation | designated reception part 118 may receive selection of 1 or more of the feature points shown in the SAR image by a user. The selection may be performed via an input / output device such as a mouse. The selected feature point is the designated feature point. The designation accepting unit 118 may accept designation of a plurality of feature points.

The designation receiving unit 118 sends information on the designated feature points to the display mode determining unit 115. The designated feature point information is, for example, an identification number or coordinates associated with each feature point.

The display mode determination unit 115 identifies candidate points related to the specified feature point. For example, the display mode determination unit 115 may cause the candidate point extraction unit 114 to extract candidate points related to the designated feature point and receive information on the extracted candidate points. Alternatively, when information that associates the feature points with the candidate points is stored in the storage unit 111, the display mode determination unit 115 may identify the candidate points based on the information.

The designation accepting unit 118 may accept designation of candidate points instead of designation of feature points. For example, the user may select any candidate point among the candidate points included in the point display image displayed by the process of step S125. The designation accepting unit 118 may accept the selection and specify a feature point related to the selected candidate point. And the designation | designated reception part 118 may identify the candidate point which concerns on the feature point.

The display mode determination unit 115 determines a display mode different from the display mode of other candidate points as the display mode of the identified candidate points. Then, the image generation unit 116 generates a point display image in which candidate points are displayed according to the determined display mode. By displaying this point display image on the display device 21, the viewer can see information on candidate points related to the designated feature point.

FIG. 18 is a diagram illustrating an example of a point display image generated by the information processing apparatus 12 according to the third modification. In FIG. 18, candidate points related to a specific feature point are displayed in white, and other candidate points are displayed in black. However, the transmittance of a figure indicating a candidate point located in the occlusion area among candidate points related to a specific feature point is 50%.

In addition, the display mode determination unit 115 displays the candidate points related to the specific feature point so that the display of the candidate points related to the specific feature point includes a display indicating the incident direction of the radar as in the second modification. Aspects may be determined.

Such a display can further suppress the possibility that the viewer misidentifies the position of the candidate point. For example, it is easily understood that the candidate points located on the wall surface of the building shown in FIG. 18 are located on a surface different from the front surface of the building.

構成 With the configuration of this modification, the viewer can easily understand the candidate points related to a specific feature point.

<< Second Embodiment >>
An information processing apparatus 10 according to an embodiment of the present invention will be described.

FIG. 19 is a block diagram illustrating a configuration of the information processing apparatus 10. The information processing apparatus 10 includes a candidate point extraction unit 104, a display mode determination unit 105, and an image generation unit 106.

Candidate point extraction unit 104 determines the target point based on the position in the three-dimensional space of the target point, which is a point specified in the intensity map of the signal from the target object acquired by the radar, and the shape of the target object Candidate points that are points that contribute to the signal at the point are extracted. The candidate point extraction unit 114 of each of the above embodiments is an example of the candidate point extraction unit 104.

The signal intensity map is, for example, a SAR image. A point specified in the intensity map is associated with a point in the three-dimensional space. An example of the target point is a feature point in the first embodiment. Note that the shape of the object to be observed is given by, for example, three-dimensional model data.

The display mode determination unit 105 displays the display mode indicating the position of the candidate point in the spatial image in which the observed object is captured based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image. decide. The shape of the subject shown in the spatial image is given by, for example, three-dimensional model data.

The display mode determination unit 115 of each of the above embodiments is an example of the display mode determination unit 105.

The image generation unit 106 generates an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode. The relationship between the points in the three-dimensional space and the points in the spatial image may be performed in advance or may be performed by the image generation unit 106. In other words, the image generation unit 106 generates an image indicating the position of the candidate point in the spatial image based on the position of the candidate point in the three-dimensional space and the relationship between the spatial image and the three-dimensional space.

The image generation unit 116 in each of the above embodiments is an example of the image generation unit 106.

FIG. 20 is a flowchart showing an operation flow of the information processing apparatus 10.

Candidate point extraction unit 104 determines the target point based on the position of the target point, which is a point specified in the intensity map of the signal from the observed object acquired by the radar, in the three-dimensional space and the shape of the observed object. Candidate points that are points that contribute to the signal at the point are extracted (step S101).

Next, the display mode determining unit 105 displays the display mode indicating the position of the candidate point in the spatial image in which the observed object is captured, based on the position of the candidate point in the three-dimensional space and the shooting condition of the spatial image. Based on the determination (step S102).

Then, the image generation unit 106 generates an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode (step S103).

According to this configuration, in the intensity map of the signal from the observed object acquired by the radar, it is easy to understand the point on the observed object that contributes to the signal at the point in the region where the layover occurs. can do. The reason is that the candidate point extraction unit 104 extracts candidate points that contribute to the signal at the target point based on the model data, and the image generation unit 106 generates an image in which the position of the candidate point in the spatial image is displayed. Because. Furthermore, the display mode of the display indicating the position of the candidate point is determined based on the position in the three-dimensional space and the imaging condition of the spatial image.

<Hardware Configuration for Implementing Each Unit of Embodiment>
As described above, in each embodiment of the present invention described above, each component of each device represents a functional unit block.

The processing of each component may be realized, for example, by reading and executing a program stored in a computer-readable storage medium that causes the computer system to execute the processing. “Computer-readable storage media” includes, for example, portable media such as optical disks, magnetic disks, magneto-optical disks, and non-volatile semiconductor memories, and ROMs (Read Only Memory) and hard disks built into computer systems. It is a storage device. "Computer-readable storage medium" is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In this case, a program or a program that temporarily holds a program such as a volatile memory in a computer system corresponding to a server or a client is also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already stored in a computer system.

The “computer system” is a system including a computer 900 as shown in FIG. 21 as an example. The computer 900 includes the following configuration.
CPU (Central Processing Unit) 901
・ ROM902
-RAM (Random Access Memory) 903
A program 904A and storage information 904B loaded into the RAM 903
A storage device 905 that stores the program 904A and storage information 904B
A drive device 907 that reads / writes from / to the storage medium 906
A communication interface 908 connected to the communication network 909
An input / output interface 910 for inputting / outputting data
-Bus 911 connecting each component
For example, each component of each device in each embodiment is realized when the CPU 901 loads the RAM 903 and executes a program 904A that realizes the function of the component. A program 904A for realizing the function of each component of each device is stored in advance in the storage device 905 or the ROM 902, for example. Then, the CPU 901 reads the program 904A as necessary. The storage device 905 is, for example, a hard disk. The program 904A may be supplied to the CPU 901 via the communication network 909, or may be stored in advance in the storage medium 906, read out to the drive device 907, and supplied to the CPU 901. The storage medium 906 is a portable medium such as an optical disk, a magnetic disk, a magneto-optical disk, and a nonvolatile semiconductor memory.

There are various modifications to the method of realizing each device. For example, each device may be realized by a possible combination of a separate computer 900 and a program for each component. A plurality of constituent elements included in each device may be realized by a possible combination of one computer 900 and a program.

In addition, some or all of the components of each device may be realized by other general-purpose or dedicated circuits, computers, or combinations thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus.

When a part or all of each component of each device is realized by a plurality of computers, circuits, etc., the plurality of computers, circuits, etc. may be centrally arranged or distributedly arranged. For example, the computer, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system and a cloud computing system.

The present invention is not limited to the embodiment described above. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Some or all of the above embodiments may be described as in the following supplementary notes, but are not limited to the following.

<< Appendix >>
[Appendix 1]
Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Candidate point extracting means for extracting candidate points that are contributing points;
A display mode in which a display mode indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image. A determination means;
Image generating means for generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
An information processing apparatus comprising:
[Appendix 2]
The display mode determination unit according to claim 1, wherein the display mode determination unit determines a different display mode depending on whether or not the candidate point is located in an area that is a blind spot when viewed from a photographic body that has captured the spatial image. Information processing device.
[Appendix 3]
The information processing apparatus according to appendix 2, wherein the display mode determination unit determines a different display mode according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image.
[Appendix 4]
The display mode determining means is further configured to display the position of the candidate point such that an indication of an incident direction of an electromagnetic wave from the radar that is a basis of the signal contributed by the candidate point. Determine the display mode of the display showing the position,
The information processing apparatus according to any one of appendices 1 to 3.
[Appendix 5]
The display mode determining means determines the display mode of the display so that a part or all of the display indicating the position of the candidate point is a transparent color for at least one of the candidate points.
The information processing apparatus according to any one of appendices 1 to 4.
[Appendix 6]
The display mode determination means further uses the position of the other candidate point in the spatial image as a display mode of display indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method. Determine a display mode different from the display mode of the display
The information processing apparatus according to any one of appendices 1 to 5.
[Appendix 7]
Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Extract candidate points that contribute,
A display mode of display indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image,
Generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
Information processing method.
[Appendix 8]
The information processing method according to appendix 7, wherein a different display mode is determined depending on whether or not the candidate point is located in a region that is a blind spot when viewed from a photographing body that has captured the spatial image.
[Appendix 9]
9. The information processing method according to appendix 8, wherein a different display mode is determined according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image.
[Appendix 10]
A display mode for indicating the position of the candidate point so that the display indicating the position of the candidate point further indicates the incident direction of the electromagnetic wave from the radar, which is the basis of the signal contributed by the candidate point. To decide,
The information processing method according to any one of appendices 7 to 9.
[Appendix 11]
The display mode of the display is determined so that a part or all of the display indicating the position of the candidate point is a transparent color for at least one of the candidate points.
The information processing method according to any one of appendices 7 to 10.
[Appendix 12]
Further, the display mode indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method is different from the display mode indicating the position of the other candidate point in the spatial image. Determine the display mode,
The information processing method according to any one of appendices 7 to 11.
[Appendix 13]
Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Candidate point extraction processing for extracting candidate points that are contributing points;
A display mode in which a display mode indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image. The decision process,
An image generation process for generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
A computer-readable storage medium for storing a program for causing a computer to execute the program.
[Appendix 14]
The display mode determination process according to appendix 13, wherein the display mode determination process determines a different display mode depending on whether or not the candidate point is located in a region that is a blind spot when viewed from the photographic body that has captured the spatial image. Storage medium.
[Appendix 15]
15. The storage medium according to appendix 14, wherein the display mode determination process determines a different display mode according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image.
[Appendix 16]
The display mode determination process is performed so that the display indicating the position of the candidate point further indicates an incident direction of an electromagnetic wave from the radar that is a basis of the signal contributed by the candidate point. Determine the display mode of the display showing the position,
The storage medium according to any one of appendices 13 to 15.
[Appendix 17]
The display mode determination process determines the display mode of the display so that a part or all of the display indicating the position of the candidate point is a transparent color with respect to at least one of the candidate points.
The storage medium according to any one of appendices 13 to 16.
[Appendix 18]
In the display mode determination process, the position of another candidate point in the spatial image is further displayed as a display mode indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method. Determine a display mode different from the display mode of the display
The storage medium according to any one of appendices 13 to 17.

10, 11, 12 Information processing device 104 Candidate point extraction unit 105 Display mode determination unit 106 Image generation unit 111 Storage unit 112 Feature point extraction unit 113 Geocoding unit 114 Candidate point extraction unit 115 Display mode determination unit 116 Image generation unit 117 Display Control unit 118 Designation receiving unit 1111 SAR data 1112 SAR data parameter 1113 Model data 1114 Spatial image 1115 Imaging condition information 21 Display device 900 Computer 901 CPU
902 ROM
903 RAM

904A program

904B storage information 905 storage device 906 storage medium 907 drive device 908 communication interface 909 communication network 910 input / output interface 911 bus

Claims

Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Candidate point extracting means for extracting candidate points that are contributing points;
A display mode in which a display mode indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image. A determination means;
Image generating means for generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
An information processing apparatus comprising:
The display mode determining unit determines a different display mode according to whether or not the candidate point is located in a region that is a blind spot when viewed from a photographing body that has captured the spatial image. Information processing device.
The information processing apparatus according to claim 2, wherein the display mode determination unit determines a different display mode according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image. .
The display mode determining means is further configured to display the position of the candidate point such that an indication of an incident direction of an electromagnetic wave from the radar that is a basis of the signal contributed by the candidate point. Determine the display mode of the display showing the position,
The information processing apparatus according to any one of claims 1 to 3.
The display mode determining means determines the display mode of the display so that a part or all of the display indicating the position of the candidate point is a transparent color for at least one of the candidate points.
The information processing apparatus according to any one of claims 1 to 4.
The display mode determination means further uses the position of the other candidate point in the spatial image as a display mode of display indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method. Determine a display mode different from the display mode of the display
The information processing apparatus according to any one of claims 1 to 5.
Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Extract candidate points that contribute,
A display mode of display indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image,
Generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
Information processing method.
The information processing method according to claim 7, wherein a different display mode is determined depending on whether or not the candidate point is located in an area where a blind spot is seen from a photographing body that has captured the spatial image.
The information processing method according to claim 8, wherein a different display mode is determined according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image.
A display mode for indicating the position of the candidate point so that the display indicating the position of the candidate point further indicates the incident direction of the electromagnetic wave from the radar, which is the basis of the signal contributed by the candidate point. To decide,
The information processing method according to any one of claims 7 to 9.
The display mode of the display is determined so that a part or all of the display indicating the position of the candidate point is a transparent color for at least one of the candidate points.
The information processing method according to any one of claims 7 to 10.
Further, the display mode indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method is different from the display mode indicating the position of the other candidate point in the spatial image. Determine the display mode,
The information processing method according to any one of claims 7 to 11.
Based on the position in the three-dimensional space of the target point, which is the point specified in the intensity map of the signal from the observed object acquired by the radar, and the shape of the observed object, the signal at the target point Candidate point extraction processing for extracting candidate points that are contributing points;
A display mode in which a display mode indicating the position of the candidate point in the spatial image in which the observed object is captured is determined based on the position of the candidate point in the three-dimensional space and the imaging condition of the spatial image. The decision process,
An image generation process for generating an image in which the position of the candidate point in the spatial image is displayed according to the determined display mode;
A computer-readable storage medium for storing a program for causing a computer to execute the program.
14. The display mode determination process according to claim 13, wherein the display mode determination process determines a different display mode depending on whether or not the candidate point is located in an area that is a blind spot when viewed from a photographic body that has captured the spatial image. Storage media.
15. The storage medium according to claim 14, wherein the display mode determination process determines a different display mode according to the number of times a half line from the candidate point to the photographic object intersects the surface of the subject in the spatial image.
The display mode determination process is performed so that the display indicating the position of the candidate point further indicates an incident direction of an electromagnetic wave from the radar that is a basis of the signal contributed by the candidate point. Determine the display mode of the display showing the position,
The storage medium according to any one of claims 13 to 15.
The display mode determination process determines the display mode of the display so that a part or all of the display indicating the position of the candidate point is a transparent color with respect to at least one of the candidate points.
The storage medium according to any one of claims 13 to 16.
In the display mode determination process, the position of another candidate point in the spatial image is further displayed as a display mode indicating the position of the candidate point contributing to the signal at the target point specified by a predetermined method. Determine a display mode different from the display mode of the display
The storage medium according to any one of claims 13 to 17.