JP5885974B2 - Corresponding point setting method, corresponding point setting device, and corresponding point setting program for aerial photo image data - Google Patents

Description

  The present invention relates to an aerial photograph image data corresponding point setting method, corresponding point setting apparatus, and corresponding point setting program for automatically detecting corresponding points corresponding to each other from aerial photograph image data obtained by photographing a feature from an aircraft in aerial surveying. About.

  Conventionally, aerial triangulation is a technique for precisely obtaining the position and orientation at which a photographic image taken for aerial photogrammetry is taken. As shown in FIG. 8, a target object from an aircraft flying at a predetermined altitude is shown. This is done by photographing the features of the area without any problem and processing the photographed image (aerial photograph image data I). The shooting by the aircraft measures the corresponding points between the aerial photograph image data I continuously photographed in the flight direction and the aerial photograph image data I photographed in the adjacent flight courses 1, 2 and 3. The information of the coordinate known points included in the photographic image data I also defines the absolute positional relationship between the image and the feature photographed by the block adjustment method such as bundle adjustment. For example, aerial photograph image data continuously photographed along the flight direction are photographed so that the overlapping ratio is 60% or more, and aerial photograph image data photographed in different flight courses have a mutual overlapping ratio. Photographed to be 30% or more. As shown in FIG. 9, captured aerial image data is often associated with an external orientation element between a shooting position measured by GPS and a camera posture with respect to a feature measured by an IMU. Since the measured external orientation elements do not have sufficient accuracy to finally perform aerial photogrammetry, it is necessary to measure corresponding points between images and perform aerial triangulation. Then, by obtaining corresponding points corresponding to the feature points extracted from the common objects included in the overlapping areas of the aerial photograph image data, and performing simultaneous adjustment measurement of the external orientation elements by a block adjustment method such as bundle adjustment The measured external orientation element is determined more precisely. By using the external orientation elements determined precisely in this way, it is possible to determine the position of the feature in the three-dimensional space from the aerial photograph image data overlapping each other based on the principle of triangulation.

JP 2003-323603 A

However, it is easy to set the corresponding points between the aerial photographic image data when the overlapping ratio is large to some extent as when taken along the flight direction, but the aerial photographic image data when adjacent in different flight courses As described above, when the overlapping ratio is small, it is easy to set the corresponding points. That is, in the aerial photograph image data, when the overlapping ratio is large, the movement distance of the aircraft is small, so the change in the appearance of the feature to be imaged is small, but when the overlapping ratio is small, the moving distance of the aircraft is large, Even if the same object is seen, the appearance of the aerial photo image data obtained by shooting is greatly different, so even if the feature point is extracted from the same object There are cases where the same feature is not recognized and is not set as a corresponding point. In addition, although the feature points are extracted from different objects, they may be recognized as the same feature and set as corresponding points because the arrangement of the feature points is similar. In particular, in a forest with a similar landscape or a mountainous area with a large slope, it is difficult to automatically set corresponding points because the appearance of the target feature changes immediately.
As described above, when the corresponding points cannot be automatically set, it is necessary for the operator to set the corresponding points by visually comparing the images, and the work requires skill and time. It was.

  The present invention provides a corresponding point setting method, corresponding point setting apparatus, and corresponding point for aerial photo image data that enable automatic setting of corresponding points between aerial photo image data with high accuracy in order to solve the above problems. Provide a point setting program.

As a first aspect of the corresponding point setting method for aerial photo image data for solving the above-described problem, common feature points are common to external orientation elements including shooting positions and shooting postures, and are common to each other in overlapping areas. The first aerial photo image data and the second aerial photo image data, the first aerial photo image data and the second aerial photo image data having a predetermined coordinate position, and the first aerial photo image data at a rate smaller than the overlapping rate. Corresponding point setting method for aerial photo image data for setting corresponding points common to each other from the data and the third aerial photo image data overlapping with the second aerial photo image data, the first aerial photo image data and the first aerial photo image data 2. Triangulation from the coordinate position of the common feature points of the aerial photo image data and the external orientation elements when the first aerial photo image data and the second aerial photo image data are taken A step of calculating the coordinate position of the common feature point in the three-dimensional space based on the principle, a straight line connecting the coordinate position when the third aerial photograph image data is captured and the coordinate position of the common feature point in the three-dimensional space A step of calculating a projection position of the common feature point intersecting the third aerial photograph image data onto the third aerial photograph image data, and a search area having a predetermined size including the projection position are set in the third aerial photograph image data. Pattern matching is performed on common feature points for the step and the search area, and the amount of displacement and the direction of displacement from the coordinate position to the projection position in the search area when the matching rate is equal to or higher than the threshold and the highest matching rate is calculated. A step of setting as an error vector, a step of detecting the most frequent error vector among error vectors, Was embodiments comprising a step of vector is set to the corresponding point corresponding to the common feature point positions obtained by adding the most frequent error vector to the projection position of the position at which the most frequent error vector.
According to this aspect, the coordinate position on the camera coordinate of the common feature point common to the first aerial photograph image data and the second aerial photograph image data whose positional relationship is specified in advance, the first aerial photograph image data, and the second aerial photograph image data Based on the principle of triangulation, the actual coordinate position of the common feature point is calculated based on the principle of triangulation from the relationship between the shooting position and shooting attitude in the 3D space stored as an external orientation element when the aerial photo image data is shot. can do. Furthermore, a straight line connecting the calculated coordinate position of the common feature point in the three-dimensional space and the photographing position in the three-dimensional space stored as an external orientation element when the third aerial photograph image data is photographed is a third aerial photograph. The projection position of the common feature point intersecting with the image data on the third aerial photograph image data is calculated, the search area including the projection position is set as the third aerial photograph image data, and the search area is defined by the projected common feature point. Pattern matching is performed, and the amount of misalignment and the misalignment direction from the position in the search region where the matching rate is the highest among the matching points with the matching rate equal to or higher than the threshold to the projection position are calculated and set as the error vector. By detecting the most frequently occurring mode error vector and adding the mode error vector to the projection position, the third aerial photograph image data position is obtained. It is possible to calculate the corresponding point corresponding to the passing characteristic point.

Further, as a second aspect of the corresponding point setting method of the aerial photo image data for solving the above-described problem, the common feature point is the first aerial photo overlapping the second aerial photo image data and the third aerial photo image data. Extracting a first sample area having a predetermined size from the area of the image data and identifying a second sample area corresponding to the first sample area from the second aerial photograph image data; and the first sample area and the second sample area A characteristic shape is extracted as a first feature point and a second feature point by a feature point extraction filter from the shape of the feature represented in the region, and the extracted first feature point is extracted from the second feature point. Then, the second feature point matching the first feature point is detected by pattern matching, and the first feature point and the second feature point are set as common feature points.
According to this aspect, the first sample region having a predetermined size is extracted from the region of the first aerial photo image data that overlaps with the second aerial photo image data, and the second sample region corresponding to the first sample region is extracted as the first sample region. (2) The feature points are extracted by a predetermined feature point extraction filter for the features represented in the first sample region and the second sample region obtained by specifying from the aerial photograph image data, and the first sample A point extracted in the region is set as a first feature point, a point extracted in the second sample region is set as a second feature point, and a common feature point common to the first sample region and the second sample region is set as the first feature point. And the second feature point, the positional relationship between the first aerial photo image data and the second aerial photo image data can be specified.

As a first configuration of the corresponding point setting device for aerial photo image data for solving the above-described problem, a common feature point common to each other in a region where external orientation elements including a shooting position and a shooting posture are linked and overlapped with each other. The first aerial photo image data and the second aerial photo image data, the first aerial photo image data and the second aerial photo image data having a predetermined coordinate position, and the first aerial photo image data at a rate smaller than the overlapping rate. An aerial photo image data corresponding point setting device for setting corresponding points common to each other from the data and the third aerial photo image data overlapping with the second aerial photo image data, the first aerial photo image data and the second aerial photo image data 2. Triangulation from the coordinate position of the common feature points of the aerial photo image data and the external orientation elements when the first aerial photo image data and the second aerial photo image data are taken Common feature point coordinate position calculating means for calculating the coordinate position of the common feature point in the three-dimensional space based on the principle, the coordinate position when the third aerial photograph image data is taken, and the coordinates of the common feature point in the three-dimensional space A feature point projection coordinate position calculation means for calculating a projection position on the third aerial photograph image data of a common feature point where a straight line connecting the positions intersects the third aerial photograph image data; and a predetermined size including the projection position The search area setting means for setting the search area to the third aerial photo image data, and the coordinates in the search area when the common feature point is pattern-matched to the search area and the matching ratio is equal to or higher than the threshold and the highest matching ratio is matched. An error vector calculation means for calculating a positional deviation amount and a positional deviation direction from the position to the projection position, and setting as an error vector; A mode error vector detecting means for detecting a mode error vector that appears frequently and a corresponding point corresponding to a common feature point at a position obtained by adding the mode error vector to the projection position where the error vector becomes the mode error vector And a corresponding point setting means for setting.
According to this configuration, the same effect as the first method can be obtained.

Further, as a second configuration of the aerial photo image data corresponding point setting device for solving the above-described problem, the common feature point is determined from a region of the first aerial photo image data overlapping with the second aerial photo image data. A sample area setting means for extracting a first sample area having a size and identifying a second sample area corresponding to the first sample area from the second aerial photograph image data, and representing the first sample area in the first sample area and the second sample area Of the shape of the extracted feature, a characteristic shape is extracted as a first feature point and a second feature point by a feature point extraction filter, and the extracted first feature point is pattern-matched to the second feature point. The second feature point that coincides with the first feature point is detected, and the first feature point and the second feature point are set by the common feature point detecting unit that sets the common feature point.
According to this configuration, the same effect as that of the second method can be obtained.

As an aspect of the corresponding point setting program for aerial photo image data for solving the above-described problem, the process according to claim 1 or 2 is executed by a computer.
According to this aspect, the same effect as the first method or the second method can be obtained.

  The summary of the invention does not list all necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

It is a schematic block diagram of a corresponding point setting apparatus. It is a figure which shows an example of the feature point extracted in the sample area | region. It is a conceptual diagram of aerial triangulation. It is a conceptual diagram which shows calculation of an error vector. It is a figure which shows the error vector group extended from a projection point. It is a figure which shows aerial photograph image data and a file number. It is a figure which shows the flowchart of the program memorize | stored in the corresponding point setting apparatus. It is an acquisition conceptual diagram of aerial photograph image data. It is a figure which shows the geometry in an aerial photograph. It is a schematic block diagram of the other form of a corresponding point setting apparatus.

  Hereinafter, the present invention will be described in detail through embodiments. However, the following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

Embodiment 1
As shown in FIG. 8, in the aerial photogrammetry, a target area α to be surveyed is divided into a plurality of regions δα having substantially equal sizes, each region δα is photographed by a camera from an aircraft, and the photographed aerial photos are processed by image processing. It is done by doing. Specifically, the image is taken from the air while flying at a predetermined altitude along a flight path composed of a plurality of courses 1, 2, and 3 as indicated by a two-dot chain line in the drawing. Courses 1, 2, and 3 are set so that the adjacent direction of courses 1, 2, and 3 is north-south in the vertical direction if the extension direction of each course 1, 2, and 3 is, for example, east-west. The area α is set to be photographed with the minimum necessary number of times. The aerial photographs are taken so that the shooting ranges overlap each other in the directions along the courses 1, 2, 3 by about 60% or more. In the direction adjacent to the image, the images are photographed so that the photographing ranges overlap at about 30% or more. In this embodiment, the extension directions of the courses 1, 2, and 3 are set as, for example, east and west, and the adjacent directions of the courses 1, 2, and 3 are set as the north and south in the vertical direction, but depending on the shape of the target area α to be surveyed, The course and the adjacent direction of the course may be set so that the target area α can be efficiently photographed.

FIG. 9 is a diagram illustrating the geometric relationship between aerial photography and the imaging position and orientation.
The aircraft is equipped with a GPS / IMU device that measures the shooting position Oi and shooting posture, and a camera that shoots a feature. The GPS antenna is attached to a predetermined position of the aircraft based on aerial photogrammetry regulations, and acquires latitude, longitude, and altitude at the time of flight from a GPS satellite. In addition, the photographing position Oi can be obtained with high accuracy by photographing five or more GPS satellites in a time zone that can be captured. The subscript i of the shooting position Oi is a number of aerial photo image data described later. In the present embodiment, description will be made assuming that i = 1 to 21 (see FIG. 6).
As shown in FIG. 9, the coordinate position (Xi, Yi, Zi) of the shooting position Oi acquired by the GPS is represented by the ground coordinate system. The ground coordinate system is represented by a Cartesian coordinate system, for example, with the east-west direction (latitude direction) as the X axis, the north-south direction (meridian direction) as the Y axis, and the height direction from the ground surface as the Z axis. Specifically, a geocentric coordinate system (a three-dimensional coordinate system centered on the geocentric center) or a three-dimensional coordinate system in which the altitude is added to the plane direction in the Gauss-Kruger projection is often used.

  The IMU is mounted on a camera used for aerial shooting and measures a shooting posture that changes depending on the flight posture of the aircraft. Specifically, the IMU includes a gyro and an acceleration sensor, and the flight attitude of the aircraft, for example, ω (rotation angle around the X axis) indicating rolling of the flight attitude, φ (rotation angle around the Y axis) indicating pitching, Κ (rotation angle around the Z axis) indicating yawing is measured. That is, by measuring the flight attitude with the IMU, the influence of the flight attitude on the shooting is recorded.

  The coordinate position (Xi, Yi, Zi) at the photographing position Oi measured by GPS and the rotation angles κi, φi, ωi around the coordinate axes by the IMU are external orientation elements for specifying the photographing position and the photographing posture. Note that the coordinate position (Xi, Yi, Zi) of the shooting position is the ground coordinate system.

The camera is mounted at a predetermined position on the aircraft. Photographing by the camera is performed so that the optical axis is substantially perpendicular to the ground surface. For example, a digital area camera that digitizes and acquires a photograph of a feature is applied to the camera.
The aerial photograph taken by the camera is stored in a storage medium 2 such as a hard disk or a memory as aerial photograph image data Ii together with an external orientation element and a focal distance ci for each photographing. Note that the focal length ci can be regarded as the same focal length when shooting with the same camera at a fixed focus.
The external orientation element and the focal length ci corresponding to the position and the shooting posture for each shooting are linked to the storage medium 2 and stored.

  Note that the camera is not limited to a digital area camera, and any camera such as a digital video camera, an analog video camera, or a film camera that can photograph a feature as a photograph may be used. Considering the time from taking a picture to processing the image of the picture, it is preferable to use a digital optical camera such as a digital area camera or digital video camera that can directly input the taken picture as image data to a computer. Moreover, a thing with small optical distortion (lens distortion) is preferable. In the following description, it is assumed that the influence of lens distortion is basically small, or image data in which such influence is corrected is used.

FIG. 1 shows a schematic configuration diagram of the corresponding point setting device 1.
With respect to the aerial photo image data Ii stored in the storage medium 2, the correspondence between the aerial photo image data Ii is set by the corresponding point setting device 1.
The corresponding point setting device 1 is a computer that executes processing for setting corresponding points that coincide with each other in overlapping portions of the aerial photograph image data Ii obtained by photographing a feature from the air. A ROM, RAM, and HDD as storage means and an interface as communication means are included, and corresponding point setting processing is executed in accordance with a program stored in the storage means. The corresponding point setting device 1 is connected to input means such as a keyboard 4 and a mouse, and display means such as a monitor 6.

The corresponding point setting device 1 includes a reading unit 11, a common feature point setting unit 10, a feature point projected coordinate position calculation unit 16, a search area setting unit 17, an error vector calculation unit 18, and a mode error vector detection unit. 19, corresponding point setting means 20, and output means 21.
The reading unit 11 reads the aerial photograph image data Ii, the external orientation elements (Xi, Yi, Zi, κi, φi, ωi) and the focal length ci from the storage medium 2.

2 (a) and 2 (b) show the first sample region G1 extracted from the aerial photograph image data I _i continuous along the flight course 1 by the feature point extraction means 13 and the aerial photograph image data I _{i + 1.} It is a figure which shows an example of the 1st and 2nd feature points Q1, Q2 extracted from the 2nd sample area | region G2 corresponding to the 1st sample area | region G1. Hereinafter, the aerial photograph image data captured first in each flight course will be described as I ₁ , and the next photographed aerial photograph image data will be described as I ₂ . Further, describing the aerial photograph image data to be duplicated in the aerial photograph image data I ₁ and the aerial photograph image data I ₂ as I _3.
The common feature point setting unit 10 includes a sample area setting unit 12, a feature point extraction unit 13, a common feature point detection unit 14, and a common feature point coordinate position calculation unit 15.
Sample area setting means 12 extracts the first sample area G1 of the aerial photograph image data I ₂ and aerial photograph image data I ₃ and overlapping aerial photographic image from the data I ₁ of the region predetermined size, the first sample A second sample area G2 corresponding to the area G1 is specified from the second aerial photograph image data. Position for extracting a first sample region G1, since it has been found overlapping ratio of the aerial photograph image data I ₁ and the aerial photograph image data I _2, aerial photographic image data I ₂ in aerial photograph image data I ₁ What is necessary is just to set so that it may extract from the overlapping side. For example, consider a pixel coordinate system of the aerial photograph image data I _1, known aerial photograph image data I ₁ and the aerial photograph image data I ₂ duplicates aerial photograph image data ranges of the pixel coordinates that overlap on the basis of the ratio of I of Set to ₁ . Next, a range of overlapping pixel coordinates is set in the aerial photograph image data I ₁ based on the overlapping ratio of the known aerial photograph image data I ₁ and the aerial photograph image data I ₃ . By setting the first sample area G1 from within this range is set from the range of the first sample area G1 aerial photograph image data I ₁ and the aerial photograph image data I ₂ and aerial photograph image data I ₃ overlap be able to.
The method for the aerial photograph image data I ₁ and the aerial photograph image data I ₂ and aerial photograph image data I ₃ sets the first sample area G1 from overlapping ranges is not limited to the above, aerial photograph image as a result data I ₁ and the aerial photograph image data I ₂ and aerial photograph image data I ₃ may be such that the first sample area G1 is set from overlapping ranges.
Further, the extraction of the second sample area G2, may be extracted by a technique such as pattern matching the position corresponding to the first sample area G1 a second aerial photograph image data I within _2.
Size of the first sample area G1, for example, when the size of the aerial photograph image data _{I 1} and 10000 pixels × 10000 pixels, is divided into about 1 / 1000-1 / 10000 of the total. In the number of pixels the aerial photograph image data I _1, when the sides people were 200 split becomes a size of 1/4000 of the original aerial photograph image data I, the magnitude of the first sample area G1 is 50 pixels × 50 Set to pixel.

The feature point extraction means 13 includes a feature point extraction filter for extracting a feature point Qi from the feature represented in the image, as indicated by a circle in FIGS. 2A and 2B, and includes aerial photograph image data I _1. the first sample area G1 and the aerial photograph image data second sample area feature represented in G2 of I _2, for example building corners, white lines of the corners of the road, and distinct features relative to the surrounding as the corner of the terrain Are extracted as first and second feature points Q1 and Q2. For example, a Forersner filter, a SUSAN filter, a FAST filter, or the like is applied as a specific feature point extraction filter.

The common feature point detector 14 detects a common feature point (q1; q2) from the first sample region G1 and the second sample region G2. Common feature point detection unit 14 is to pattern matching a second feature point Q2 included a plurality of first feature points Q1 included in the first sample area G1 of the aerial photograph image data I ₁ and the second sample area G2 To detect a common feature point (q1; q2) which is a common feature point. In the pattern matching, the first feature point Q1 is corrected in a state where the first sample region G1 is corrected by the rotation angles (κ1, φ1, ω1) and the second sample region G2 is corrected by the rotation angles (κ2, φ2, ω2). And a common feature point (q1; q2) common to the second feature point Q2 is detected. That is, the first sample area G1 and the common feature points detected from the second sample area G2 (q1; q2) is a corresponding point of the aerial photograph image data I ₁ and the aerial photograph image data I _2.

Figure 3 is a diagram showing a relationship between the aerial photograph image data I ₁ and the aerial photograph image data I ₂ and aerial photograph image data I ₃ in a three-dimensional space.
The common feature point coordinate position calculation means 15 is the actual feature of the common feature point (q1; q2) detected by the common feature point detection means 14, the coordinate position (xQ, yQ, zQ) of the point Q in the ground coordinate system. Is calculated. Common feature point; position of (q1 q2), since each aerial photograph image data I ₁ of the photographic coordinates and aerial photograph image data I ₂ pictures coordinates are known, from the photograph coordinate system with the focal length c1 and c2 Convert to camera coordinate system and convert from camera coordinate system to ground coordinate system using external orientation elements (X1, Y1, Z1, κ1, φ1, ω1) and (X2, Y2, Z2, κ2, φ2, ω2) Thus, the coordinate position (xQ, yQ, zQ) on the ground coordinates is calculated.

Hereinafter, a method for calculating the coordinate position (xQ, yQ, zQ) of the common feature point (q1; q2) in the ground coordinate system by the common feature point coordinate position calculation unit 15 will be described in detail. Since the feature point q1 and the feature point q2 are obtained by photographing the same point Q of the feature, the coordinate position (xQ, yQ, zQ) of the point Q from the feature point q1 and the feature point q2 is as follows. Is calculated.
Coordinate position in photographic coordinate system of the feature point q1 in aerial photograph image data _{I 1} is (x1, y1), a coordinate position in the photographic coordinate system of the feature point q2 in aerial photograph image data _{I 2} is represented as (x2, y2) . The coordinate position (x1, y1) of the feature point q1 and the coordinate position (x2, y2) of the feature point q2 are actually projected on the imaging plane separated by the focal lengths c1, c2 from the imaging center of the camera. Therefore, if the coordinate positions of the photographing positions O ₁ and O ₂ are set as the photographing center, and the photographing centers O ₁ and O ₂ are the origins of the respective camera coordinate systems, the feature point q1 in the camera coordinates is the coordinate position (x1, y1, -c1) and the feature point q2 are represented as coordinate positions (x2, y2, -c2). The coordinate positions of the shooting positions O ₁ and O _{2 in} the ground coordinate system are given by an external orientation element at the time of shooting. Specifically, the shooting position O ₁ is the coordinate position (X1, Y1, Z1), and the shooting position. O ₂ is the coordinate position (X2, Y2, Z2).

Therefore, Q (xQ, yQ, zQ) is a forward intersection method based on the relationship between the photographing center O ₁ of the aerial photograph image data I ₁ , the photographing center O ₂ of the aerial photograph image data I ₂ , and the feature points q 1 and q _2. From the principle of triangulation such as For example, Q (xQ, yQ, zQ) can be obtained by the following equation (1).
_{u1 · (Q-O 1)} · u1-Q = - (u2 · (Q-O 2) · u2-Q) ... formula (1)
^{u1 = R1 -1 · q1 / |} q1 | ... formula (2)
^{u2 = R2 -1 · q2 / |} q2 | ... formula (3)
Here are those u1 is shown in equation (2) representing a unit vector directed toward the feature point q1 from the photographing position O ₁ in the ground coordinate system, Equation (3) u2 shown in the feature point from the captured position O ₂ q2 The unit vector heading to is represented in the ground coordinate system. R1 and R2 are transformation matrices (rotation matrices) for converting the coordinate system from the camera coordinate system to the ground coordinate system set by the rotation angles (κ1, φ1, ω1) and (κ2, φ2, ω2) representing the photographing posture. R1 ^-1 and R2 ^-1 are transformation matrices for coordinate transformation from the ground coordinate system to the camera coordinate system, and are inverse matrices of R1 and R2.

Feature point projected coordinate position calculation means 16, the common feature point; calculating the coordinate position (x3, y3) of the photograph coordinate system of the projection point q3 when (q1 q2) is projected to aerial photograph image data I ₃ . Specifically, the coordinate position of the point Q, which is calculated by the common feature point coordinate position calculation means 15 (xQ, yQ, zQ) and the coordinate position of the taking center _{O 3} and (X3, Y3, Z3), the projection point q3 It is calculated based on the collinear condition with the coordinate position.
Common feature points (q1; q2) projected point q3 in aerial photograph image data _{I 3} which is projected, when the coordinate position in the camera coordinate system (x3, y3, z3) is, (x3, y3, z3) = R3 · It is calculated from the relationship (Q−O ₃ ). R3 is the rotation angle of the external orientation parameters representing the photographing position in the photographing center O3 (κ3, φ3, ω3) coordinate converting points on aerial photographic image data I ₃ from the camera coordinate system which is set to the ground coordinate system by transformation It is a matrix (rotation matrix). That is, regarded as a position vector coordinate position of the projection point q3 and (x3, y3, z3) the imaging center _{O 3} as the origin of the camera coordinate system, a position vector (x3, y3, z3) a real number times the Z-direction When the coordinate position coincides with the focal distance c3, the coordinate position of the projection point q3 in the camera coordinates is x3, y3 in the coordinate position (x3, y3, -c3) of the projection point q3 on the camera coordinates is
x3 = −c3 · (x3 / z3)
y3 = −c3 · (y3 / z3)
It can ask for. That is, the coordinate position of the photograph coordinate system of the projection point q3 by projection onto the aerial photograph image data I ₃ of the line segment connecting the imaging center O ₃ and the projection point q3 (x3, y3) is calculated. Accordingly, the coordinate position (x3, y3, −c3) of the projection point q3 in the camera coordinate system is calculated.

Thus, the calculated coordinate position of the projected point q3 (x3, y3, -c3) is collinear with the projected point q3, the imaging center _{O 3,} the coordinate position of the point Q (xQ, yQ, zQ) and although those mathematically calculated from the condition, the measured external orientation parameters, i.e., the coordinate position of the taking center O ₃ upon shooting aerial photograph image data _{I 3 (X3, Y3, Z3} ) and photographing posture rotation angle indicating the (κ3, φ3, ω3) since the error is contained, common feature points in the aerial photograph image data I ₃ (q1; q2) to the error is caused for the correct position of the corresponding point corresponding Yes. In case the measurement accuracy of the exterior orientation elements is sufficiently high, the coordinate position in the photographic coordinate system of the projection point q3 aerial photograph image data I ₃ calculated by the feature point projected coordinate position calculation means 16 (x3, y3) is common A minute shift occurs compared to the position where the feature points (q1; q2) correspond correctly. It is known that this minute shift appears as a parallel displacement displaced in the same direction from a position considered to be correct at the projection point q3 where all the common feature points (q1; q2) are projected.

In the search area setting unit 17, FIG. 4 (a), the (b), the by the feature point projected coordinate position calculation means 16 of a predetermined aerial photograph image data I ₃ in the projected projection point q3 around the size of the A search area T is set. The search area T is set, for example, as a rectangular area having a predetermined size so that the projection point q3 is located at the center. The size of the search region T is a size that does not exceed the measurement accuracy of the external orientation element, in other words, the coordinate position (x3, y3) of the projection point q3 in the photographic coordinate system, and the projection point q3 is aerial photographic image data. It is set to be larger than the deviation of the minute of the positions corresponding correctly; (q2 q1) common feature points in I _3.
In addition, if the measurement accuracy of the external orientation elements acquired at the time of photographing the aerial photograph image data I ₁ , I ₂ , I ₃ is high, the size of the search region T is small, and if the measurement accuracy is uneasy, the search is slightly performed. It is better to set the area larger.

The error vector calculation means 18 performs pattern matching on the search region T including the projection point q3 with the pattern of the feature point q1 or the feature point q2 of the projected common feature point (q1; q2), and a matching rate is set in advance. The search region T is searched for a position where the matching rate is the maximum among the points that match at or above the threshold value. In the pattern matching, the pattern of the feature point q1 or the feature point q2 is corrected by the rotation angle (κ1, φ1, ω1) or rotation angle (κ2, φ2, ω2) of the external orientation element, respectively, and the search region T is rotated. This is performed for those corrected by (κ3, φ3, ω3). Then, the distance and direction between the position where the matching rate is maximized and the position of the projection point q3 are calculated. As the matching rate, a position at which a higher value is output as the pattern matching is better is used. For example the pixels surrounding the feature point q1 in aerial photograph image data I _1, may be calculated a correlation coefficient between the pixels surrounding a certain pixel within the search area T. Then, the position at which the matching rate becomes the maximum with a value equal to or greater than the threshold value is defined as the coordinate position (x4, y4) of the correction point q4, and the coordinate position (x4, y4) of the correction point q4 and the coordinate position of the projection point q3 ( The positional deviation amount and the positional deviation direction with respect to x3, y3) are calculated. In other words, the error vector calculation means 18 uses the error vector Verr = (x4−x3) from the coordinate position (x3, y3) of the projected projection point q3 to the coordinate position (x4, y4) of the correction point q4 that seems to be most correct. , Y4−y3). The error vector has a projection point q3 as a start point and a correction point q4 as an end point. Then, all the common feature points common to the aerial photograph image data I ₁ and I _2; by calculating the error vector Verr against (q1 q2), as shown in FIG. 5, a plurality of error vectors Verr group calculate.

  The mode error vector detection means 19 searches the error vector Verr group calculated by the error vector calculation means 18 for the error vector Verr that occurs most frequently in the search region T as the mode error vector Vopt. When the error of the external orientation element is small, it can be mathematically derived that this error becomes a certain amount in a certain direction in the local region in the image. In other words, it can be predicted that a correctly matched portion has the same error vector. Therefore, it can be determined that the mode error vector Vopt is matched correctly.

Corresponding point setting means 20 corresponds to the common feature point (q1; q2) the position obtained by adding the mode error vector Vopt to the projection point q3 having the same error vector as the mode error vector Vopt, that is, the position of the correction point q4. Calculated as the coordinate position (x5, y5) of the corresponding point q5. That is, the calculated corresponding point q5 is set as a corresponding point common to the aerial photograph image data I ₁ , I ₂ , and I ₃ .

  The output means 21 includes the coordinate position (x5, y5) of the corresponding point q5 calculated by the corresponding point setting means 20, the coordinate position (x1, y1) of the first feature point Q1 corresponding thereto, and the second feature point. The coordinate position (x2, y2) of Q2 is linked and output to the corresponding point database 7 as a storage medium.

Therefore, the common feature points on aerial photographic image data I ₁ and I _2; seems to project the (q1 q2) corresponding points is the aerial photograph image data I _3, using the error vector Verr, statistically correct By correcting the mode error vector Vopt by adding it to the projection point q3, the coordinate position (x5, y5) of the corresponding point q5 corresponding to the aerial photograph image data I ₁ and I ₂ can be set automatically. . In addition, an error vector Verr group is calculated from the projection point q3 group projected for each of the plurality of common feature points (q1; q2) extracted from the aerial photograph image data I ₁ and I ₂ , and the error vector Verr group is the highest among the error vector Verr groups. Detecting a frequent mode error vector Vopt, in other words, detecting a statistically most frequent mode error vector Vopt in the error vector Verr group, and using the mode error vector Vopt, the aerial image data I _{1 is} aerial since to correct the deviation of the projection point q3 photographic image data I _3, even with a small overlap with the aerial photograph image data I ₁ and the aerial photograph image data I _3, corresponding to each other in the aerial photograph image data I _1, I ₃ Corresponding points can be reliably identified.

Hereinafter, an embodiment of a method for setting corresponding points by the corresponding point setting device 1 will be described with reference to flowcharts.
The photographed aerial photograph image data Ii (i = 1 to 21) includes the course number m (m = 1 to 3) and the photograph numbers n (n = 1 to 7) photographed in the courses 1, 2, and 3. Two numbers are assigned as file numbers, and are stored in the storage medium 2 as aerial photo image data files m_n. Specifically, as shown in FIG. 6, for example, in the case of course 1, files 1_1, 1_2,..., In the case of course 2, files 2_1, 2_2,. Files 3_1, 3_2, and so on are attached.

  When the corresponding point setting device 1 is activated, a screen is displayed by a program stored in the storage means. On the screen, an aerial photograph image data file m_n to be processed is loaded, a processing variable setting button, and the like are arranged and set by a mouse or a keyboard. When the input setting is completed, the file name for starting the setting of the corresponding points, the corresponding focal length, and the external orientation element are displayed as numerical values on the screen. When the start of setting corresponding points is instructed, first, the aerial photo image data Ii stored in all the aerial photo image data files m_n is read by the reading means 11 (step 101).

The following processing will be described assuming that it is managed by the file number m_n.
First, the course number m and the shooting number n of the file to be processed are reset (step 102). Next, the course number m is set to 1 (step 103), and then the shooting number n is set to 1 (step 104). Next, a file to be processed is set according to the set course number m and shooting number n. For example, file m_n, file m_n + 1 adjacent to the flight direction, file m + 1_n overlapping file m_n and file m_n + 1 in different flight courses, and file m + 1_n + 1 overlapping file m_n and file m_n + 1 are set (step 105). Specifically, the aerial photo image data I ₁ corresponding to the file 1_1, the aerial photo image data I ₂ corresponding to the file 1_2, and the aerial photo image data corresponding to the file 2_1 are started from m = 1 and n = 1. Setting of corresponding points with I ₁₄ and corresponding points between aerial photo image data I ₁ corresponding to file 1_1, aerial photo image data I ₂ corresponding to file 1_2, and aerial photo image data I ₁₃ corresponding to file 2_2 The setting is performed (see FIG. 6). That is, in this embodiment, the region where the aerial photo image data I ₁ , I ₂ and I ₁₄ overlap and the region where the aerial photo image data I ₁ , I ₂ and I ₁₃ overlap are regarded as substantially the same. it is possible, aerial photograph image data I ₁₄ and common feature point set in the aerial photograph image data I ₁ and I ₂ that overlaps the common and I ₁₃ (q1; q2) aerial photograph image data I using the corresponding points between ₁₄ and I ₁₃ were to be calculated.

Next, a first sample area having a predetermined size is selected from areas overlapping the aerial photograph image data I ₂ , aerial photograph image data I _14, and aerial photograph image data I ₁₃ of the aerial photograph image data I ₁ by the sample area setting means 12. extract the G1, by identifying the second sample area G2 corresponding aerial photograph image data I ₂ performs pattern matching in a first sample area G1 with respect to the aerial photograph image data I ₂ in the first sample area G1, aerial photograph image data I _1, sets a corresponding sample area to one another in the aerial photograph image data I ₂ (step 106).

Plurality Next, the feature point extraction filter characteristic point extraction unit 13, a first feature point Q1 from the feature of a plurality represented in aerial photograph image data I _1, from the feature represented in aerial photograph image data I ₂ The second feature point Q2 is extracted. Extracting a plurality of first feature points Q1 and the second feature point Q2 were a coordinate position in each photograph coordinates of the aerial photograph image data _{I 1} and the aerial photograph image data _{I 2} (x1, y1) and the coordinate position (x2, y2 ) Is stored (step 107).

Next, the common feature point detection means 14 performs pattern matching of the plurality of first feature points Q1 with the plurality of second feature points Q2 to detect the common feature points (q1; q2). In other words, the common feature point detection unit 14 detects a common point Q appearing in the aerial photograph image data I ₁ and the aerial photograph image data I ₂ (step 108).

Next, the common feature point coordinate position calculation means 15 calculates the coordinate position (xQ, yQ, zQ) of the point Q in the ground coordinate system of the common feature point (q1; q2) (step 109). Specifically, the common feature point; is one with a focal length c1 of the time of photographing aerial photograph image data I ₁ including the first feature point Q1, the coordinate position of the photograph coordinate system (q1 q2) (x1 , the coordinate position of the camera coordinate system with its origin at the photographing position _{O 1} from y1) (x1, y1, converted to -c1), the rotation angle of the exterior orientation elements (.kappa.1, .phi.1, the transformation matrix R1 obtained from .omega.1) The camera coordinate system is converted to the coordinate position (x1g, y1g, -c1g) of the ground coordinate system. The common feature point; other with focal length c2 is an internal orientation parameters at the time of photographing of the aerial photograph image data I ₂ including one second feature point Q2, the coordinate position of the photograph coordinate system (q1 q2) ( x2, y2) coordinates of the camera coordinate system to the imaging position _{O 2} as the origin from (x2, y2, converted to -c2), the rotation angle of the exterior orientation elements (κ2, φ2, ω2) transformation matrix obtained from R2 The camera coordinate system is converted to the coordinate position (x2g, y2g, -c2g) of the ground coordinate system.

Then, the photographing position O ₁ (X1, Y1, Z1) measured as an external orientation element as the coordinate position of the ground coordinate system, the coordinate position (x1g, y1g, −c1g) of the first feature point Q1, and the coordinate position of the point Q (XQ, yQ, ZQ), the coordinate position (X2, Y2, Z2) of the shooting position O2 measured as an external orientation element as the coordinate position of the ground coordinate system, and the coordinate position of the second feature point Q2 ( x2g, y2g, -c2g) and the coordinate position (xQ, yQ, zQ) of the point Q in the ground coordinate system are calculated by the principle of triangulation from the coordinate position (xQ, yQ, ZQ) of the point Q.

Next, in order to call the aerial photo image data I ₁₄ or the aerial photo image data I ₁₃ that overlaps the aerial photo image data I ₁ and the aerial photo image data I ₂ , the image number n is replaced with the control variable k. (Step 110).
Next, the call of the aerial photograph image data I ₁₄ by the control variable k in step 110, the feature point projected coordinate calculating unit 16, the common feature point; when (q1 q2) is projected to aerial photograph image data I ₁₄ The projection coordinate position of the projection point q3 in the photographic coordinate system is calculated (step 111). Specifically, the imaging position _{O 14} common feature points upon shooting aerial photograph image data _{I 14} in the ground coordinate system; coordinates of point Q, which is calculated by (q1 q2) (xQ, yQ , zQ) and straight line to calculate the coordinate position (x3, y3) in photographic coordinate system of the projection point q3 is an intersection at which intersects the aerial photograph image data I ₁₄ connecting.

Next, the search area setting means 17 sets a search area T having a predetermined size around the projection point q3 (step 112).
Next, the error vector calculation means 18 pattern-matches the projected feature point Q1 (Q2) itself as the template F with respect to the search region T including the projection point q3 (step 113), and the matching rate is equal to or greater than the threshold value. A coordinate position when matching is performed at the highest matching rate is detected from the search region T as a correction point q4 (step 114). Then, a displacement amount and a displacement direction between the position of the correction point q4 and the position of the projection point q3 are calculated and output as an error vector Verr (step 115).

Next, the most frequent error vector detection means 19 detects the error vector Verr that occurs most frequently in the search region T from the error vectors Verr as the most frequent error vector Vopt (step 116).
Next, the corresponding point setting means 20 sets the position obtained by adding the mode error vector Vopt to the projection point q3 where the error vector Verr becomes the mode error vector Vopt, and the corresponding point q5 corresponding to the common feature point (q1; q2). The coordinate position (x5, y5) is calculated (step 117). That is, the corresponding point q5 is set to be common to the aerial photograph image data I ₁ , I ₂ , and I ₁₄ .
Next, it is determined whether or not the control variable k for calling the aerial photograph image data is equal to the shooting number n + 1. If the value of the control variable k is different from the shooting number n + 1, the process proceeds to step 119 and the value of the control variable k is set. Is equal to the shooting number n + 1, the process proceeds to step 120 (step 118).
If the value of the control variable k is different from the shooting number n + 1, the process proceeds to step 119, where the value of the control variable k is updated and replaced with the control variable k + 1 (step 119), and the process returns to step 111, and the aerial photograph of the file 2_2 The processing of the image data I ₁₃ is repeated from step 111 to step 117, and corresponding points common to the aerial photograph image data I ₁ , I ₂ , I ₁₃ are set.
Next, if it is determined in step 118 that the photographing number is equal to n + 1, the control variable k is reset to 0 (zero) in step 120 (step 120).
Next, it is determined whether or not the shooting number n is smaller than 6. If the shooting number n is smaller than 6, the process returns to step 104, the shooting number n is updated to n + 1, and steps 105 to 120 are performed. Step 104 to Step 120 are repeated until the shooting number n reaches 6. If the shooting number n is 6, the process proceeds to step 122 (step 121).
Next, it is determined whether or not the course number is smaller than 3. If the course number is smaller than 3, the process returns to step 103, the shooting number m is updated to m + 1, and steps 104 to 121 are performed. Steps 103 to 121 are repeated until the course number m becomes 3. If the course number m is 3, the corresponding points set in all the aerial photo image data I that overlap each other are output to the corresponding point database 7 via the output means 21 and the process ends. 122).

Note that the aerial photograph image data is not limited to the image of the feature recorded by the digital area camera, but may be digitalized by a film scanner from a film photographed by an analog camera.
In the above embodiment, as shown in FIG. 6, the aerial photograph image data I ₁ and the aerial photograph image data I ₂ which overlap each other in the course direction, aerial photographic image data I ₁ and the aerial photograph image data in the course alignment direction set the corresponding points that are common to the aerial photograph image data I ₁₄ or aerial photographic image data I ₁₃ I ₂ overlap at a smaller rate than the rate of overlap with respect to aerial photograph image data I ₁ and the aerial photograph image data I ₂ As described above, according to the above method, the aerial photo image data I ₃ and the aerial photo image data I ₁ and the aerial photo image data I ₁ and the aerial photo image data I ₂ that overlap each other in the course direction are combined. it is also possible to set the corresponding points in common with each other from a region overlapping with the picture image data I _2.

In the above embodiment, the corresponding point setting device 1 includes the common feature point configured by the sample area setting unit 12, the feature point extraction unit 13, the common feature point detection unit 14, and the common feature point coordinate position calculation unit 15. Although it has been described that the setting unit 10 is provided and sets common feature points that correspond to the aerial photograph image data I ₁ and the aerial photograph image data I ₂ that overlap in the flight direction, the present invention is not limited to this, but aerial photograph image data using the epipolar geometry between I ₁ and aerial photograph image data I _2, the common feature point; it may be set to (q1 q2).

Embodiment 2
In the first embodiment, the corresponding point setting device 1 has been described as including the common feature point setting unit 10. However, the second embodiment is different in that the common feature point setting unit 10 is not provided. That is, as shown in FIG. 10, in the aerial photograph image data I ₁ and the aerial photograph image data I ₂ that are adjacent in the courses 1, 2, and 3 in advance, the aerial photograph image data I ₁ and the aerial photograph image data I ₂ If the coordinate position in the three-dimensional space of the common feature point (q1; q2), which is the corresponding point of, is previously stored in the storage unit 2, the corresponding point setting device 1 reads the reading unit 11 and the feature point projected coordinate position calculation. Means 16, search area setting means 17, error vector calculation means 18, mode error vector detection means 19, corresponding point setting means 20, and output means 21 may be used. Thus, the common feature point is a corresponding point of the aerial photograph image data I ₁ and the aerial photograph image data I _2; if (q1 q2) is set, the air stored in the storage unit 2 by reading means 11 Photo image data Ii, external orientation elements, and common feature points are read, feature point projected coordinate position calculation means 16, search area setting means 17, error vector calculation means 18, mode error vector detection means 19, and corresponding point setting. by treating means 20, as in embodiment 1, it is possible to set a common corresponding points q5 to the aerial photograph image data I ₁ and the aerial photograph image data I ₂ and aerial photograph image data I _3.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above embodiment.

1 corresponding point setting device, 11 reading means, 12 sample area setting means,
13 feature point extraction means, 14 common feature point detection means,
15 common feature point coordinate position calculating means, 16 feature point projected coordinate position calculating means,
17 search area setting means, 18 error vector calculation means,
19 mode error vector detection means, 20 corresponding point setting means, 21 output means,
I Aerial photo image data, Q; Q1; Q2 feature points, G1; G2 sample area,
T search area, α target area,
Verr error vector, Vopt mode error vector.

Claims (5)

First aerial photo image data and second aerial photo image data in which external orientation elements including a shooting position and a shooting posture are linked, and coordinate positions of common feature points that are common to each other in an overlapping region are preset; Third aerial photo image data that overlaps the first aerial photo image data and the second aerial photo image data at a rate that is smaller than the rate at which the first aerial photo image data and the second aerial photo image data overlap. Corresponding point setting method of aerial photo image data for setting corresponding points common to each other,
Triangulation from the coordinate position of the common feature point of the first aerial photo image data and the second aerial photo image data and the external orientation element when the first aerial photo image data and the second aerial photo image data are taken. Calculating a coordinate position in a three-dimensional space of the common feature points based on a surveying principle;
The third feature point of the common feature point at which a straight line connecting the coordinate position when the third aerial photograph image data is captured and the coordinate position of the common feature point in the three-dimensional space intersects the third aerial photograph image data. Calculating a projection position on the aerial photo image data;
Setting a search area of a predetermined size including the projection position in the third aerial photo image data;
The common feature point is pattern-matched to the search region, and the amount of displacement and the direction of displacement from the coordinate position to the projection position in the search region when the matching rate is equal to or higher than the threshold and the highest matching rate is obtained. Calculating and setting as an error vector;
Detecting the most frequent error vector among the error vectors;
A corresponding point setting method for aerial photographic image data including a step of setting a position obtained by adding a mode error vector to the projection position where the error vector becomes a mode error vector as a corresponding point corresponding to the common feature point . The common feature point is obtained by extracting a first sample area having a predetermined size from an area of the first aerial photograph image data that overlaps the second aerial photograph image data, and a second sample corresponding to the first sample area. Identifying a region from the second aerial photo image data;
A characteristic shape is extracted as a first feature point and a second feature point by the feature point extraction filter from among the shapes of the features represented in the first sample region and the second sample region, respectively, Pattern matching of one feature point with the second feature point to detect a second feature point matching the first feature point, and setting the first feature point and the second feature point as a common feature point The corresponding point setting method for aerial photo image data according to claim 1, which is set by: First aerial photo image data and second aerial photo image data in which external orientation elements including a shooting position and a shooting posture are linked, and coordinate positions of common feature points that are common to each other in an overlapping region are preset; Third aerial photo image data that overlaps the first aerial photo image data and the second aerial photo image data at a rate that is smaller than the rate at which the first aerial photo image data and the second aerial photo image data overlap. Corresponding point setting device for aerial photo image data for setting corresponding points common to each other,
Triangulation from the coordinate position of the common feature point of the first aerial photo image data and the second aerial photo image data and the external orientation element when the first aerial photo image data and the second aerial photo image data are taken. Common feature point coordinate position calculating means for calculating a coordinate position of the common feature point in a three-dimensional space based on the principle of surveying;
The third feature point of the common feature point at which a straight line connecting the coordinate position when the third aerial photograph image data is captured and the coordinate position of the common feature point in the three-dimensional space intersects the third aerial photograph image data. A feature point projected coordinate position calculating means for calculating a projection position on the aerial photo image data;
Search area setting means for setting a search area of a predetermined size including the projection position in the third aerial photo image data;
The common feature point is pattern-matched to the search region, and the amount of displacement and the direction of displacement from the coordinate position to the projection position in the search region when the matching rate is equal to or higher than the threshold and the highest matching rate is obtained. Error vector calculating means for calculating and setting as an error vector;
A mode error vector detecting means for detecting a mode error vector that occurs most frequently among the error vectors;
Corresponding points of aerial photographic image data comprising corresponding point setting means for setting a position obtained by adding a mode error vector to the projection position where the error vector becomes a mode error vector as a corresponding point corresponding to the common feature point Setting device. The common feature point is obtained by extracting a first sample area having a predetermined size from an area of the first aerial photograph image data that overlaps the second aerial photograph image data, and a second sample corresponding to the first sample area. Sample area setting means for specifying an area from the second aerial photo image data;
A characteristic shape is extracted as a first feature point and a second feature point by the feature point extraction filter from among the shapes of the features represented in the first sample region and the second sample region, respectively, Commonly setting one feature point to a second feature point to detect a second feature point that matches the first feature point, and setting the first feature point and the second feature point as a common feature point 4. The corresponding point setting device for aerial photograph image data according to claim 3, which is set by the feature point detecting means.   A corresponding point setting program for aerial photographic image data that causes a computer to execute the processing according to claim 1 or 2.

Images