JPWO2008152740A1 - Digital aerial 3D measurement system - Google Patents

Abstract

An object of the present invention is to provide a digital aerial photography three-dimensional measurement system capable of calculating three-dimensional coordinates with high accuracy from an image photographed by a digital camera. The present invention inputs an internal orientation element based on the specifications of a digital camera, a plurality of aerial photographs taken with the digital camera, and GPS data received along with the aerial photography, to a computer, geometric correction means, and continuity A corrected photograph is output from a computer by means of analysis, tie point extraction means, external orientation means, deviation correction means, stereo matching means, and ortho correction means. It was set as the structure of the digital aerial photography three-dimensional measurement system.

Description

  The present invention is a system related to photogrammetry performed from an aircraft using a digital camera as an imaging device.

Photogrammetry is a technique for calculating a three-dimensional coordinate of an object based on information about the camera position and orientation when the object is photographed from a plurality of positions and angles and these photographs are taken. Among photogrammetry, aerial photogrammetry is a method that calculates the latitude, longitude, and altitude of a feature or terrain by shooting the direction directly from the flying aircraft. Especially, it is not a chemical film but an electronic film as a photo recording medium. What uses traditional media is called digital aerial photogrammetry.
Usually, the image recorded on the imaging surface of the camera is obtained by projecting a range that falls within the angle of view of the camera by central projection. That is, when a straight line passing through the principal point of the lens is extended from each point on the imaging surface, the first collision of the straight line is recorded at that point.
Therefore, the three-dimensional coordinates of each point on a photo follow the optical system in reverse if you know the optical system of the camera that took the photo and the position and orientation of the imaging surface when the photo was taken. It is somewhere on each straight line obtained. At this time, if there is another photo of the same object taken at different positions and orientations, straight lines corresponding to each point can be obtained in the same way, but the same point is used for the previous photo and the subsequent photo. The straight lines obtained for are always different and must intersect. That is, the intersection is the desired coordinate.
Conventional aerial photogrammetry often uses a large-scale system developed exclusively for its use, and it has high accuracy, but is expensive to introduce and operate, and is difficult unless the project has a large budget. However, the consumer digital camera market is also growing rapidly, and there are cameras with high resolution that can withstand photogrammetry.
As described in Japanese Patent Application Laid-Open No. 2007-108029, which is Japanese Patent Application Laid-Open No. 2007-108029, based on an error amount obtained by comparing imaging data captured by the imaging unit and geographic data corresponding to the imaging range of the imaging unit at each point. An invention has also been disclosed in which positioning can be performed even under conditions where the number of GPS satellite supplements cannot be sufficiently secured by specifying the position by the position specifying unit.
As described in Japanese Patent Application Laid-Open No. 2006-079521, which is Japanese Patent Application Laid-Open No. 2006-079521, a plurality of image information captured by a digital camera from a plurality of points with different shooting positions for each shooting target, and a three-dimensional image of the shooting target The coordinate information is used to obtain image coordinate information on the image information corresponding to the three-dimensional coordinate information at the predetermined position of the photographing object, and the RGB value corresponding to the obtained image coordinate information is obtained to obtain the image of the photographing object. An invention for forming an orthographic projection image is also disclosed.
As described in Japanese Patent Application Laid-Open No. 2001-114047, which is Patent Document 3, when an image from an arbitrary viewpoint is synthesized using an image from a camera installed in a car, a road surface is synthesized by a road surface index synthesis unit. An invention is also disclosed in which a projected image is combined with an index for distinguishing an object other than a road surface so that the driver can easily judge an object other than the road surface.
As described in Japanese Patent Application Laid-Open No. 2006-195758, which is Patent Document 4, an image input unit that acquires two stereo images whose horizontal axes are parallel projections, and two images obtained by the image input unit Among them, an image collating unit that calculates a similarity between the corrected image obtained by deforming a local region in one image and another image, and a three-dimensional point cloud based on the similarity calculated by the image collating unit An invention provided with a three-dimensional point cloud generation means for generating is also disclosed.
As described in Japanese Patent Laid-Open No. 2004-038660, which is Patent Document 5, a plurality of windows associated with a base image and a reference image based on a predetermined correlation are set. The stereo image matching between the reference image window and the reference image window can be achieved by eliminating the separation or overlap between adjacent windows in the reference image by expanding and contracting a plurality of windows formed in the reference image. An invention for performing and restoring the shape of an object is also disclosed.
However, since the conventional dedicated system is entirely composed of hardware, the entire apparatus is likely to be large, and maintenance is troublesome. Moreover, even if the technology improves, it is difficult to partially replace the technology.
In addition, since the invention described in Japanese Patent Application Laid-Open No. 2007-108029, which is Patent Document 1, uses a sky image camera to capture the entire periphery with infrared rays, dedicated hardware is required, and consumer digital It cannot be realized with a camera.
The invention described in Japanese Patent Application Laid-Open No. 2006-079521, which is Patent Document 2, forms an orthographic projection image from three-dimensional coordinate information of an object to be imaged, and images the object to be imaged from a plurality of points with a digital camera. Therefore, the three-dimensional coordinates are not calculated.
The invention described in Japanese Patent Application Laid-Open No. 2001-114047, which is Patent Document 3, combines images captured by a camera mounted on a vehicle and displays the situation around the vehicle in an easy-to-understand manner. Thus, high-precision three-dimensional coordinates are not calculated.
The invention described in Japanese Patent Application Laid-Open No. 2006-195758, which is Patent Document 4, is a method in which two cameras are fixed, photographed while moving in parallel and stereo-matched, and the position and orientation of the camera are indefinite. It is not something that can be handled.
The invention described in Japanese Patent Application Laid-Open No. 2004-038660, which is Patent Document 5, divides a standard image obtained by photographing the same object and a reference image into the same number of windows, respectively, and performs stereo matching according to the correspondence between the windows. It does not calculate the correspondence between each point over the entire region where two images overlap, and calculate high-accuracy three-dimensional coordinates.
Therefore, an object of the present invention is to provide a digital aerial imaging three-dimensional measurement system that can calculate three-dimensional coordinates with high accuracy from an image photographed by a digital camera.

  In order to solve the above problems, the present invention inputs an internal orientation element based on the specifications of a digital camera and a plurality of aerial photographs taken by the digital camera to a computer, and uses the internal orientation element as a coefficient and radial distortion. Geometric correction means for storing geometrically corrected photograph data obtained by converting the coordinates on the image of the aerial photograph with respect to the tangential distortion, and reading the aerial photograph from the storage apparatus and receiving the aerial photograph along with the photographing GPS data is input to a computer, the difference between two consecutive photographs of the aerial photograph is calculated to calculate the difference in shooting position based on the altitude information of the GPS data, and the latitude and longitude information of the GPS data Means for continuity analysis for storing in a storage device the data of the camera position associated with each photo in comparison, the geometrically corrected photo and the Read out the camera position from the storage device, and based on the camera position, when adjacent photos of the geometrically corrected photos are paired with each other, check where the multiple points on one photo appear in the other photo Tie point extracting means for storing the tie point data found by the above in a storage device, the internal orientation element, the geometrically corrected photograph, the camera position, and the tie point are read from the storage device, and the tie point is the camera. Position / posture in which the position is adjusted so that the twist between the two straight lines in the pair of photos is minimized by calculating a straight line that reverses the central projection in real space, assuming that the image was taken in the position directly below the position. The external orientation means for storing the data in the storage device, the internal orientation element, the aerial photograph, and the position and orientation are read from the storage device, and the navigation Deviation for storing in a storage device the data of a deviation-corrected photo that has been re-projected so that the photograph is geometrically corrected with the internal orientation element and photographed in a posture directly below based on the position and orientation When the correction means, the position and orientation, and the displacement-corrected photos are read from the storage device and the overlapping photos of the displacement-corrected photos are paired, a plurality of points on one photo are the other photos The image is searched for where the image is, and a straight line that reverses the central projection in the real space is calculated based on the position and orientation, and the data of the three-dimensional coordinates obtained from the two straight lines in the pair of photographs is stored. Stereo matching means stored in the apparatus, the internal orientation element, the aerial photograph, the position and orientation, and the three-dimensional coordinates are read from a storage device, and the one represented by the three-dimensional coordinates is Correspondence when photographing in the position and orientation with the geometric distortion of the part orientation element is calculated, and each point on the aerial photograph is connected to the three-dimensional coordinate, and the image is orthogonally projected from the central projection. A digital aerial three-dimensional measurement system characterized by comprising an orthocorrection means for outputting corrected photo data converted into a digital image from a computer, and image collation by tie point extraction or stereo matching of photogrammetry In the pixel value acquisition, a frame set in a square shape when the point is near the center and a shape extended in the radial direction when the point is near the outer edge of the image, the camera traveling direction Rotate according to the camera tilt, distort according to the camera tilt, or scale according to the camera altitude, change the pixel value acquisition position according to the shape of the frame, and In the image collation method characterized in that the pixel value is calculated based on the distance between the camera and the camera posture estimation performed using the coplanar condition and the collinear condition of the photograph in which the photographing ranges overlap in the external orientation of the photogrammetry A posture estimation method characterized in that estimation is performed by using two consecutive photographs along a traveling direction of the image and a photograph taken at a position where the photograph and the photographing range overlap and are orthogonal to the traveling direction; When correcting the central projection in the direct downward direction with the position correction, for the central projection photo tilted by the camera tilt at the time of taking the photo, rotate it around the lens principal point so that the imaging surface is horizontal, and before the rotation After re-projecting each point on the imaging surface of the camera onto the rotated imaging surface, re-sampling with the orientation of the sides of each photo aligned and stereo matching for photogrammetry Painting A stereo matching method characterized in that, in finding a point on the other image corresponding to each of the above points, a stage of processing accuracy and a threshold are set, and the three-dimensional coordinates of each point are obtained by an inductive procedure And realized.

  FIG. 1 is a flowchart showing a processing flow of a digital aerial imaging three-dimensional measurement system according to the present invention, and FIG. 2 is a diagram showing a process of the digital aerial imaging three-dimensional measurement system according to the present invention, the relationship between data and apparatus, FIG. 3 is a diagram showing the configuration of a digital aerial photography three-dimensional measurement system computer according to the present invention, FIG. 4 is a diagram showing graphic deformation in the image correlation method of the digital aerial photography three-dimensional measurement system according to the present invention, FIG. 5 is a diagram showing the deformation of a figure in the image correlation method of the digital aerial imaging 3D measurement system according to the present invention, and FIG. 6 is a diagram showing the acquisition of pixel values of the digital aerial imaging 3D measurement system according to the present invention; FIG. 7 is a diagram showing coplanar conditions and collinear conditions, FIG. 8 is a diagram showing a set of images in position and orientation estimation of the digital aerial imaging three-dimensional measurement system according to the present invention, and FIG. 9 is a diagram showing the present invention. A digital sky FIG. 10 is a diagram showing a method of correcting the deviation of the three-dimensional measurement system, FIG. 10 is a diagram showing resampling of the deviation corrected image of the digital aerial photography three-dimensional measurement system according to the present invention, and FIG. 11 is the present invention. The figure which shows the method of the stereo matching of a digital aerial photography three-dimensional measurement system, The twelfth is a perspective view of the consumer digital camera used with the digital aerial photography three-dimensional measurement system which is this invention, FIG. 13 is the digital which is this invention FIG. 14 is a front view of a consumer digital camera used in the digital aerial photography three-dimensional measurement system according to the present invention, and FIG. 14 is a plan view of the consumer digital camera used in the aerial photography three-dimensional measurement system.

Hereinafter, a digital aerial imaging three-dimensional measurement system according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a flowchart showing a processing flow of the digital aerial imaging three-dimensional measurement system according to the present invention. FIG. 2 is a diagram showing the relationship between processing, data, and apparatus of the digital aerial imaging three-dimensional measurement system according to the present invention. FIG. 3 is a diagram showing a configuration of a digital aerial imaging three-dimensional measurement system computer according to the present invention.
As shown in FIG. 1, the digital aerial photography three-dimensional measurement system 1 includes geometric correction 2, continuity analysis 3, tie point extraction 4, external orientation 5, displacement correction 6, stereo matching 7, and ortho correction 8. It becomes more.
In the geometric correction 2, a photograph in which distortion aberration included in each photograph is corrected is generated using the parameters obtained in the internal orientation 9.
In the continuity analysis 3, each photograph is shuttered in synchronization with the GPS signal, but specifically, which photograph corresponds to which GPS record is determined. When a reference photograph and a GPS record corresponding thereto are input, the remaining image and the GPS record are collated and analyzed based on the photograph.
In tie point extraction 4, data for performing external orientation 5 is generated. Specifically, with respect to photographs in which a part of the shooting range overlaps, a characteristic point appearing in common is searched using image matching technology, and the coordinates on the image are stored as data.
In the external orientation 5, the position and orientation of the imaging surface of the camera at the time of capturing each photograph are estimated with high accuracy based on the position data measured by the GPS and the photograph taken by the camera.
Specifically, assuming that each photograph was taken in a direction directly below from the position as recorded in GPS, the initial condition is that rays obtained by tracing the optical system backward from the tie point 4a are essentially on the same plane. The position and orientation of the imaging surface are calculated by a solution such as Newton's method so that the torsion between each light ray that should become is minimized. In addition, when a feature (ground reference point) whose geographical coordinates are known in advance is included in the photograph, the information can be used for this estimation, but it is not essential.
In the displacement correction 6, in order to simplify the calculation of the stereo matching 7, a photograph re-projected to an image that should be taken when the image is taken in the direct downward direction is generated. Specifically, by using the result of the external orientation 5 of each photograph, the coordinates on the image of each point on the photograph are converted to a position that should be projected when the photograph is taken in the direct downward direction, and stereo matching is performed. Resample the photos with the same orientation.
In the stereo matching 7, three-dimensional information of a portion that is common to two images having a large overlapping range is calculated among a plurality of photographs after the internal orientation 9 and the external orientation 5 are applied and the displacement correction 6 is performed.
Specifically, the correspondence between points on each photo is obtained using image matching technology (where the point in one is reflected in the other), and the optical system is traced in reverse for each corresponding point. The intersection of the obtained straight lines is derived and its coordinates are obtained. Since the photograph is a raster image, this processing is performed by obtaining a correspondence relationship for each pixel.
Since the photograph is a central projection, when an object whose depth (high altitude) is found appears in the vicinity of the outer edge of the photograph, an area that cannot be seen in the shadow (falling down) occurs. Since the range hidden by the fall varies depending on the shooting position and angle, it differs from photo to photo, and stereo matching 7 cannot be performed for this range. Therefore, when it is desired to obtain three-dimensional information without a hole, photographs from various positions or angles are required so as to compensate each other for falling in each photograph.
In the ortho correction 8, the central projection is reprojected into an orthogonal projection. Although the photograph is a central projection, the coordinates of each point on the photograph are obtained as a result of stereo matching 7, so after mapping each point to this coordinate, the depth axis (altitude axis) is crushed and the vertical and horizontal axes If it is only, it will be possible to obtain an orthographic photo. A three-dimensional photograph map can be obtained by generating three-dimensional data based on the coordinates obtained as a result of the stereo matching 7 and using a photograph subjected to ortho correction 8 as a texture.
As shown in FIG. 2, the geometric correction 2 means inputs the data of the internal orientation element 9b and the aerial photograph 10a, and outputs the data of the geometrically corrected photograph 2a. Using the value of the internal orientation element 9b as a coefficient, a radial distortion equation and a tangential distortion equation are determined, and the inverse transformation is applied to the aerial photograph 10a. Specifically, it is a process of transforming the coordinates on the image by inverse transformation of the distortion and resampling it into a raster image by an interpolation method such as cubic interpolation.
The data of the internal orientation element 9b is created by the processing of the internal orientation 9 that receives the data of the camera specification 9a. Of the data of the camera specification 9a, there are a parameter that defines the optical system of the consumer digital camera 14 and a parameter that represents the distortion of the optical system determined by the internal orientation 9 processing.
The data of the aerial photograph 10 a is taken by performing the aerial photography 10. The electronic image file conforms to the output format of the consumer digital camera 14. Each photo is taken so that 60% or more of the respective shooting range overlap with the previous and subsequent photos, and at least one of the flight routes is from a non-consecutive flight route (ie crossing or parallel running route). Photographs are taken such that 30% or more of the photographed ranges overlap each other.
The data of the geometrically corrected photograph 2a is an electronic image file obtained by removing distortion inherent to the optical system of the consumer digital camera 14 from each of the data of the aerial photograph 10a.
The data of the camera specification 9a is a set of data having attributes such as the main specifications of the camera and the lens of the consumer digital camera 14, that is, the principal point position, the focal length, the pixel size, and the pixel interval.
The work of internal orientation 9 measures and corrects the distortion of the optical system of the consumer digital camera 14, that is, the error between the ideal central projection determined by the camera specification 9a and the actual central projection. Determine the parameters to be used. The central projection in photography is ideally determined by the camera's principal point position, focal length, pixel dimensions, and pixel spacing, but the decision formula itself uses approximate conditions, and the work accuracy Because of this problem, an actual lens is not ideal, and it is necessary to measure distortion aberration and the like in combination and determine parameters for correcting an error from an ideal central projection.
Specifically, for a photograph taken of a plurality of objects whose absolute coordinates are known from a position where the absolute coordinates are known, the coordinates on the image where the objects should appear under ideal conditions ( (The above calculation formula) and the coordinates on the image in which those objects are actually reflected are applied to the equation on the coordinate transformation on the image representing the radial distortion and the tangential distortion. A coefficient corresponding to an equation for reproducing the distortion of the optical system is determined. The photographs taken with the optical system are all inversely transformed by using the on-image coordinate transformation formula determined by giving the coefficients determined in this way to the distortion formula, so that they are all equally under ideal conditions. Can be converted into images taken with
In the aerial photography 10, the consumer digital camera 14 and the aviation GPS receiver 16 are mounted on the aircraft via the mounting / control base 15, and the trajectory of the aircraft is recorded using the functions of the aviation GPS receiver 16. However, the shutter signal synchronized with the recording of the aviation GPS receiver 16 is sent to the consumer digital camera 14 using the function of the mounting / control base 15, and the consumer signal is sent using the function of the mounting / control base 15. While the digital camera 14 is maintained in a posture facing directly downward, a photograph of the direct downward direction is taken from the aircraft by the function of the consumer digital camera 14.
The consumer digital camera 14 is a commercially available single-lens reflex digital camera or the like. An interface for receiving a shutter signal from the outside is provided and connected to the mounting / control base 15. The mounting part is versatile and can be replaced when a higher performance camera appears.
The mounting / control base 15 includes a consumer digital camera 14 and an aviation GPS receiver 16, and is a base that is attached to an aircraft. It absorbs vibrations derived from aircraft and has a function of tilting the mounting portion with a motor so that the consumer digital camera 14 always faces directly below. It also has a function of receiving a signal from the aviation GPS receiver 16 and recording it on a built-in memory card, and simultaneously sending a shutter signal to the consumer digital camera 14 in synchronization with the signal reception.
The aviation GPS receiver 16 is a receiver for three-dimensionally determining the position of the aircraft from the arrival time of radio waves of time signals emitted from satellites. The electronic board is built in the mounting / control base 15, and only the antenna is attached to an appropriate place outside the aircraft or inside the aircraft where radio waves are easily received, and the antenna and the board are connected by a cable.
As shown in FIG. 2, the means of continuity analysis 3 inputs aerial photograph 10a and GPS data 12a, and outputs data of camera position 3a. When one reference photo in the aerial photograph 10a and a GPS record corresponding to the photo in the GPS data 12a are input, the photograph and the GPS record are first linked. Thereafter, one photograph that is not associated with the GPS record and is continuous with the photograph associated with the GPS record is taken out from the aerial photograph 10a. Several points are selected from one photo, and where they appear on the other photo are searched by image collation by the image correlation method, and the degree of overlap of these two images is obtained. Based on this overlap and the altitude information of the GPS record associated with one of the photos, the difference between the shooting positions is converted into an actual distance, and compared with the latitude / longitude information of the GPS data 12a, The record of the GPS data 12a that should correspond to the geographical coordinates is linked. This is recursively repeated, and GPS records corresponding to each photograph of the aerial photograph 10a are extracted from the GPS data 12a and linked to obtain a camera position 3a.
The data of the GPS data 12a is data in which the accuracy of the aviation GPS data 10b in which the flight route is recorded at the time of the aerial photography 10 is improved by performing the GPS analysis 12. Data such as GPS time, geographic coordinates, and moving speed are included.
The data of the camera position 3a is data in which records (GPS time, geographic coordinates, moving speed, etc.) corresponding to the moment of shooting are linked from the GPS data 12a for each of the data of the aerial photograph 10a.
The processing of the GPS analysis 12 performs RTK-GPS analysis using the aerial GPS data 10b as mobile station data and the fixed-point GPS data 11a as base station data, and calculates the flight path of the aerial imaging 10 with higher accuracy.
The aviation GPS data 10 b is data obtained by recording the flight path when the aerial photography 10 is performed by the aviation GPS receiver 16. Data such as GPS time, geographical coordinates, moving speed, and carrier wave are included.
The data of the fixed point GPS data 11a was previously determined by installing the fixed point GPS receiver 17 at a point where the geographical coordinates are known in advance, and the data observed during or including the aerial shooting 10 implementation period The geographic coordinates of the point. Data such as GPS time and carrier wave are included.
The fixed-point positioning 11 is performed by installing a fixed-point GPS receiver 17 at a point that is close to the flight path of the aerial photography 10 (the entire flight path is preferably within a radius of 10 km) and whose geographical coordinates are known in advance. Positioning is performed during a period that is the same as or includes the aerial photography 10 implementation period. If an electronic reference point managed by the Geospatial Information Authority of Japan exists near the flight path of aerial photography 10, the observation by the electronic reference point satisfies the requirements of fixed point positioning 11, and data provided from the electronic reference point Can be used as fixed-point GPS data 11a.
The fixed-point GPS receiver 17 is a receiver that is installed in a place where the geographical coordinates are known in advance, and records the carrier wave at the geographical coordinates from the arrival time of the radio wave of the time signal emitted from the satellite.
As shown in FIG. 2, the means of tie point extraction 4 inputs the data of the geometrically corrected photograph 2a and the camera position 3a, and outputs the data of the tie point 4a. Also, the internal orientation element 9b can be input. From the relationship of the photo-position information of the camera position 3a, the overlapping photos of the geometrically corrected photos 2a are paired. Whether or not they overlap is determined by whether or not the distance between the camera positions 3a corresponding to each photograph exceeds a threshold given from the outside. When the internal orientation element 9b is input, this threshold value is calculated by estimating the size of the shooting range from the angle of view of the internal orientation element 9b and the altitude of the camera position 3a. Next, select several points from one photo for each pair, find out where they are on the other photo by image matching using the image correlation method, and record the correspondence between the coordinates on each image. To do.
The data of the tie point 4a is data in which the coordinates on each image of the points that are shown in common with respect to pairs of adjacent photographs in the data of the geometrically corrected data 2a are recorded.
As shown in FIG. 2, the external orientation 5 means inputs the data of the geometrically corrected photograph 2a, the camera position 3a, the tie point 4a and the internal orientation element 9b, and outputs the data of the position and orientation 5a. The GCP data 13b data can also be input. For the coordinates on the image of the tie point 4a, the central projection element 9b is used to reverse the central projection, and the incident light ray on the imaging surface is calculated. The incident light at the corresponding point on the photo should be in the same plane. Therefore, by using a solution such as Newton's method, the position and orientation information of the imaging surface at the time of each photo shoot is calculated by calculating the position and orientation of each imaging surface when the twist of each corresponding ray is minimized. And
In addition, when using the GCP data 13b, the incident ray is similarly calculated for each point on the photograph corresponding to the point of the GCP data 13b, and the geographical coordinates associated with the point and the ray are calculated using the above solution. The position and orientation information of each imaging surface is calculated so that the distance between the image pickup surfaces becomes the shortest at the same time. In this case, since it is necessary to change the on-image coordinates of the GCP data 13b in accordance with the geometrically corrected photograph 2a, the GCP data 13b is subjected to pre-processing similar to the geometric correction 2 using the internal orientation element 9b.
The data of the position and orientation 5a is data of the position and orientation of the imaging surface of the camera at the moment when each of the geometrically corrected photos 2a is taken.
The data of the GCP data 13b is data in which the coordinates on the image and the geographic coordinates of the photograph in which the point of the GCP information 13a is reflected in the aerial photograph 10a are linked.
The processing of the GCP designation 13 searches for the point corresponding to the location on the image of the aerial photograph 10a from the state of the ground reference point of the GCP information 13a, and determines the geographical coordinates (latitude, longitude, altitude) of the ground reference point. Record the image linked to the coordinates on the image.
The data of the GCP information 13a is recorded in the aerial photograph 10a by the aerial photograph 10 in the photographing target area of the aerial photograph 10 and is easily read on the aerial photograph 10a (uses a clue for image interpretation). Landscape photograph etc.) and geographical coordinate data of the point.
As shown in FIG. 2, the means for the displacement correction 6 inputs the data of the internal orientation element 9b, the aerial photograph 10a, and the position and orientation 5a, and outputs the data of the displacement corrected photograph 6a. From the position and orientation of the position / orientation 5a, the photograph of the geometrically corrected photograph 2a is re-projected so as to be in a shooting state when the camera is facing directly below.
Since the geometrically corrected photograph 2a is an image that has been resampled once, if the processing is performed again, there is a risk that the definition of the image may be lost. Therefore, the internal orientation element 9b is used in the aerial photograph 10a. Recalculate from where geometric correction 2 is performed. That is, the processing here converts the coordinates on the image by inverse transformation of the distortion, cancels the inclination of the position / orientation, converts the coordinates on the image in the direct downward direction, and rasters by an interpolation method such as cubic interpolation. Processing to resample to an image.
The data of the displacement-corrected photo 6a is an electronic image file obtained by projecting each of the photos of the geometrically corrected photo 2a into a photo when taken in the downward direction. Actually, in order to avoid a lack of definition, the geometrically corrected photograph 2a is not directly processed, but is reprocessed from the aerial photograph 10a where the distortion inherent to the optical system is removed.
As shown in FIG. 2, the stereo matching unit 7 inputs the data of the internal orientation element 9b, the displacement corrected photograph 6a, and the position and orientation 5a, and outputs the data of the three-dimensional coordinate 7a. Of the displacement-corrected photos 6a, overlapping photos are selected based on the position information of the position and orientation 5a and are paired. With respect to the shooting range that is commonly included in the pair of photos, the image collation method is used to find out where each point on one photo appears on the other photo. Thereafter, for each point on each photograph, a straight line that reversely follows the central projection from the real space is calculated based on the position and orientation of the position and orientation 5a. Since the data of the position and orientation 5a is obtained through the processing of the external orientation 5, the distance between the straight line extending from a certain point in one photograph and the straight line extending from the same point in the other photograph is 0, or Short. Therefore, the midpoint of the distance is set as the geographic coordinate of the point. The correspondence between the image coordinates obtained in this way and the geographic coordinates is recorded.
The data of the three-dimensional coordinate 7a is data in which the coordinates on the image and the geographic coordinates of the points appearing in common with other photographs are associated with each of the photographs of the displacement corrected photograph 6a.
As shown in FIG. 2, the orthocorrection means 8 inputs the data of the internal orientation element 9b, the aerial photograph 10a, and the three-dimensional coordinate 7a, and outputs a corrected photograph 8a. When the terrain represented by the geographical coordinates of the three-dimensional coordinate 7a is photographed by the position and orientation 5a with the geometric distortion of the internal orientation element 9b, each point on the imaging surface is projected from which point. Is calculated as the correspondence between image coordinates and geographic coordinates. Based on this correspondence, geographic coordinates are linked to each point on the image of the aerial photograph 10a, and each image coordinate is converted from a central projection to an orthographic projection based on the geographic coordinates. The re-sampled coordinates on the image are resampled into a raster image by an interpolation method such as cubic interpolation.
The data of the corrected photograph 8a is an electronic image file obtained by projecting the range of each of the aerial photographs 10a in common with the other photographs to the orthogonal projection.
Each means of the digital aerial photography three-dimensional measurement system 1 is realized by a computer 18. As shown in FIG. 3, the computer 18 includes an input device 18a, a central processing unit 18b, a storage device 18c, an output device 18d, and the like.
The input device 18a is a device that takes in the input data 18e into the computer 18 from the outside. The input data 18e is stored in the storage device 18c according to an instruction from the central processing unit 18b.
The input data 18e is data such as the internal orientation element 9b, the aerial photograph 10a, the aviation GPS data 12a, and the GCP data 13b. It may be provided in a state of being recorded on a recording medium or the like.
The central processing unit 18b is a device that analyzes instructions and executes and controls processing. If necessary, the data is taken out from the storage device 18c or stored in the storage device 18c.
The storage device 18c is a device that records programs, input data 18e, internal data 18f, output data 18g, and the like, and may be constructed as a database.
The internal data 18f is data created in the computer 18 in the course of processing. There are data such as a geometrically corrected photograph 2a, a camera position 3a, a tie point 4a, a position and orientation 5a, a displacement corrected photograph 6a, and a three-dimensional coordinate 7a.
The output device 18d is a device that extracts output data 18g from the computer 18 to the outside. The output data 18g is called from the storage device 18c according to an instruction from the central processing unit 18b.
The output data 18g is data such as the corrected photograph 8a. It is output by means such as displaying on a screen, printing on paper, or recording on a recording medium.
4 and 5 are diagrams showing the deformation of the figure in the image correlation method of the digital aerial three-dimensional measurement system according to the present invention. FIG. 6 is a diagram showing acquisition of pixel values of the digital aerial imaging three-dimensional measurement system according to the present invention.
A technique for detecting where two or more digital images appear on one image appear on another image is called image verification. In particular, in photogrammetry, for each point on each photo recorded as a raster image, which point on the other photo corresponds to that point is detected using an image correlation method. In the present invention, this method is used by means of tie point extraction 4 and stereo matching 7.
Assume that two digital images are image A and image B. Usually, for the original image A, when one point A to be subjected to image matching is arbitrarily selected, the point A is surrounded by a square frame A having a predetermined length side, The pixel value A of each point A included in the frame A is acquired.
Next, a frame B of the same size is placed on the other image B, the pixel value B of the point B included therein is acquired, and a correlation coefficient with the pixel value A in the frame A acquired previously is obtained. . While the frame A on the image A is fixed, the correlation coefficient is obtained one after another while moving the frame B on the other image B little by little, and the center point B of the frame B when the correlation coefficient becomes the highest is , The point on the image A coincides with the center point A of the frame A.
As shown in FIG. 4, in the present invention, the shape of the frame 20a placed on the image 19 is deformed. Even if they are the same, the point 21 in the vicinity of the outer edge of the image 19 is tilted in comparison with the point 20 in the center, so that the surrounding features are stretched in the radial direction. Recorded above. Therefore, if the position is near the center of the image 19, a square frame 20a is formed. If the position is near the outer edge, a square frame 21a extended in the radial direction is formed.
In the tie point extraction 4, the displacement correction 6 is not performed and the orientations of the individual images are not aligned. Therefore, as shown in the upper part of FIG. Rotate 20a.
Next, as shown in the middle part of FIG. 5, the frame 20 a is distorted in accordance with the inclination with respect to the direction directly below the camera. The tilt information used at this time can be input from the outside, but if it is not input, it is treated as having no tilt. Further, as shown in the lower part of FIG. 5, the frame 20a is enlarged or reduced in accordance with the camera height. The altitude information used at this time can be obtained from the camera position 3a.
Further, as shown in FIG. 6, as the frame 20a is deformed, the position of obtaining the pixel value 20b for calculating the correlation coefficient is also changed. In the square frame 20a without deformation, it is considered that the pixel value 20b is acquired from the center of each pixel, and the position of each center point is changed in accordance with the deformation of the frame 20a.
When acquiring the pixel value 20b from the changed acquisition position, if the acquisition position coincides with any pixel center, the pixel value 20b of the pixel is used as it is.
Next, when the acquisition position is on a straight line connecting the centers of two consecutive pixels, values obtained by interpolating the pixel values 20b of these pixels according to the distance between the center and the acquisition position are used.
In other cases, values obtained by interpolating the pixel values 20b of the four nearest pixels surrounding the acquisition position according to the distance between the center and the acquisition position are used. The expression used for the interpolation follows an interpolation method such as cubic interpolation.
In order to determine the geographical coordinates of each point on the photograph, information on the position and orientation of the camera when each photograph is taken is required. Since the position and orientation of the image pickup apparatus at the time of taking each photograph are accurately obtained to some extent based on the conventional measurement data, the position and orientation at the time of taking each photo are estimated using this as an initial value. In the present invention, this method is used by means of the external orientation 5.
The upper part of FIG. 7 shows coplanar conditions. The coplanar condition is a property that, for two photographs in which the same point is observed, a point on each imaging surface corresponding to that point and each lens principal point exist on the same plane. It is.
In the conventional method, a point 23a on the image pickup surface 23 corresponding to the point 22a on the image pickup surface 22 is searched for by using an image correlation method for two photographs whose shooting ranges overlap each other. The geographical coordinates of the point 22a and the point 23a can be expressed by an equation using the position and orientation of the camera as variables. At this time, the point 22a, the lens principal point 22b, the point 23a, and the lens principal point 23b are They are on the same plane. If many pairs of corresponding points between two photographs are found, by combining these equations and conditions, an error included in the position and orientation can be estimated and the error can be removed.
The lower part of FIG. 7 is a diagram showing collinear conditions. Note that the collinear condition is a property that a point in the photographing range, a point on the imaging surface corresponding to the point, and a lens principal point exist on the same straight line.
In the estimation method based on the coplanar condition, the variation in the position and orientation information between the images can be removed. However, when the entire image includes the same error, it cannot be removed. In particular, an error derived from the degree of freedom of rotation about a straight line connecting the lens principal points of two photographs to which the coplanar condition is applied may remain largely. Therefore, in the conventional method, a point where the geographical coordinates that can be seen in any one of the photographs is known as the ground reference point, the ground reference point 24c in the shooting range 24b, the point 24a recorded on the imaging surface 24, An error included in the position and orientation information of the photograph is estimated using the collinear condition for the lens principal point 24d, and the error is removed.
In the present invention, by using three or more photographs, an estimation accuracy equivalent to that of the conventional method is realized from only the measured position data. That is, only the position data derived by RTK-GPS is used without using the actual measurement data of the posture and without using the ground reference point.
FIG. 8 is a diagram showing a set of images in the position and orientation estimation of the digital aerial photography three-dimensional measurement system according to the present invention. In the aerial photography 10, an image 25b is prepared that is photographed at a position orthogonal to these traveling directions so that the photographing ranges overlap with two consecutive images 25 and 25a along the traveling direction.
As shown in the upper diagram of FIG. 8, first, an image 25 and an image 25a are obtained by flying while flying on a straight line, and then an image 25b is obtained by a flight path 25c that is shot while flying in parallel with the sides. Alternatively, as shown in the lower part of FIG. 8, first, images 25b and 25a are captured while flying on a straight line, and then an image 25b is captured by a flight path 25d that is captured while flying in a direction perpendicular to the straight line. Get.
Using the internal orientation element 9b, central projection is performed on the on-image coordinates of the tie point 4a obtained by applying the tie point extraction 4 to each two pairs of the images 25, 25a, and 25b thus obtained. , And calculating the incident light ray on the imaging surface at the time of each photo shooting, the incident light ray to the point on one photo of each pair and the incident light ray to the corresponding point on the other photo are: Since this is a straight line connecting each point on the imaging surface where the same point is observed and each lens principal point, it should be on the same plane due to coplanar conditions. Therefore, as an initial condition, it is assumed that each of these images is taken at the position of the position information of the camera position 3a and the camera is directed in the direct downward direction, and a solution such as Newton's method is used to solve these three pairs. If the position and orientation of each imaging surface are calculated so that the twisting of the corresponding light rays between them is minimized, this becomes equal to the position and orientation of the imaging surface at the time of each picture taking. At this time, since the lens principal points of the three photographs are not on the same straight line, the degrees of freedom of rotation around the straight lines connecting the lens principal points for these three sets of coplanar conditions cancel each other out. The error due to the degree is negligible. Therefore, it is possible to estimate the position and orientation information of the camera with very high accuracy without using the collinear condition. In addition to these three images, if there is an image that overlaps any of these three images, it is possible to estimate the position and orientation information of the image by applying the coplanar condition to that image as well. At the same time, the position and orientation information of the original three sheets is also estimated with high accuracy. Similarly, the number of images to be estimated can be increased inductively. That is, when a group of three or more images satisfies the conditions for performing the estimation by this method, if an image not included in the group overlaps with any image included in the group, the image is also In addition to the group, it can be a target of estimation by this method. Further, when a point of the GCP data 13b is reflected in any of these images, the center projection is traced backward using the internal orientation element 9b, the incident light ray at the point is calculated, and the solution is used. Information on the position and orientation of each imaging plane is calculated so that the twist of each light ray is minimized and the distance between the geographical coordinates associated with the point and the light ray is also shortest. In this case, since it is necessary to change the on-image coordinates of the GCP data 13b in accordance with the geometrically corrected photograph 2a, the GCP data 13b is subjected to pre-processing similar to the geometric correction 2 using the internal orientation element 9b.
In the present invention, prior to stereo matching 7, displacement correction 6 is applied to each photograph. That is, a process of re-projecting the central projection photograph tilted by the tilt of the camera at the time of taking a picture to the central projection in the direct downward direction is performed.
FIG. 9 is a diagram showing a deviation correction method of the digital aerial photographing three-dimensional measurement system according to the present invention. At low latitudes, the latitudes and meridians locally form square grids, but all of the photographed photographs 26 taken with the camera tilted are distorted grids, so information on the camera tilt Is used to correct the deviation so as to form a square grid, and the deviation-corrected photograph 27 is generated.
First, the imaging surface 26a inclined with respect to the imaging range 26b is rotated around the lens principal point 26c so that the light rays from the imaging range 26b remain horizontal. Next, with respect to each point on the image before rotation, the position of each point is moved to a position where each corresponding light beam intersects with the rotated imaging surface 27a.
FIG. 10 is a diagram showing a resample of an image subjected to displacement correction in the digital aerial imaging three-dimensional measurement system according to the present invention. In order to convert each point moved by the leveling 29 of the photograph 28 into a raster image, a resample process is performed and recorded as a displacement corrected image. At this time, in order to simplify the subsequent calculation of the stereo matching 7, re-sampling is performed so that the directions of the sides of the images are aligned between the images to be stereo matched. In particular, in the case of a series of photographs, the image is rotated 29a so that the sides of the resampled image are aligned with the aircraft traveling direction 28a at the time of each photographing.
For two or more digital images, on the basis of image collation, the coordinates on the image where one image appears on one image and the coordinates on the image where the same image appears on another image are obtained, and By taking the position and orientation of the camera into consideration, the three-dimensional coordinates of the camera can be obtained. In the present invention, this method is used by means of stereo matching 7.
The position and orientation of the camera when the two images having a wide range shown in common are photographed by the position and orientation estimation method are obtained. Then, for each point on one image, a corresponding point is found from the other image by the image correlation method.
FIG. 11 is a diagram showing a stereo matching method of the digital aerial photographing three-dimensional measurement system according to the present invention. The equation of the straight line 30d extending from the point 30a on one imaging surface 30 to the imaging range 30c through the lens principal point 30b in the reverse direction can be uniquely obtained. Similarly, the equation of the straight line 31d extending from the corresponding point 31a on the other imaging surface 31 to the imaging range 31c through the lens principal point 31b can be obtained uniquely.
Since the point 30a and the point 31a are projections of the same object point by different central projection systems, the straight line 30d and the straight line 31d should intersect, and the result of solving the equations of these straight lines 30d and 31d simultaneously Corresponds to the coordinate of the intersection point, that is, the three-dimensional coordinate 32. If the straight lines 30d and 31d do not intersect, the midpoint of the two straight lines is obtained and used as the three-dimensional coordinate 32.
In the present invention, for two overlapping images A and B subjected to the displacement correction 6, a point 31a on the image B corresponding to each point 30a on the image A is found by an inductive procedure. A three-dimensional coordinate 32 is obtained.
Step 0
The number of stages of processing accuracy is n, and threshold values th (1) to th (n) and threshold values diff (1) to diff (n) are determined. The moving point Xa is placed on the image A and the moving point Xb is placed on the image B, and the initial position and the direction in which they are moved are determined.
I-th procedure (i = 1 to n)
(First step)
Among the points on the image A, the moving point Xa is placed at a point closest to the initial position with respect to the direction in which the moving point is moved among the points that have not yet been determined in the 0th to (i-1) steps. Also, the moving point Xb is placed at the initial position of the image B.
(Second step)
The correlation coefficient cc between the frame Wa and the frame Wb is obtained by the image correlation method with the frames centering on the moving point Xa and the moving point Xb, respectively, as Wa and Wb. If the correlation coefficient cc is greater than or equal to the threshold th (i), the process proceeds to the third step, and otherwise, the process proceeds to the fourth step.
(Third step)
The small range Sa including the moving point Xa is taken, and for each point included in the small range Sa, the correlation coefficient between the frame Ws and the frame Wb centered on that point is calculated, and the point having the highest correlation coefficient is calculated. Let Xa ′. If the distance between the moving point Xa and the point Xa ′ on the image is less than or equal to the threshold value diff (i), the process proceeds to the fifth step, and otherwise the process proceeds to the fourth step.
(4th step)
The moving point Xb is moved to the next point of the image B, and the process returns to the second step. If there is no next point, the process proceeds to the sixth step.
(5th step)
A point corresponding to the moving point Xa is defined as Xb.
(6th step)
The moving point Xa is moved to the next point on the image A, and the process returns to the second step. If there is no next point, the process proceeds to the (i + 1) th procedure.
Step (n + 1)
The three-dimensional coordinates are obtained for the points for which the correspondence is determined in the 0th to nth procedures.
Step (n + 2)
For the points for which the correspondence relationship has not been determined in the 0th to nth steps, enter a value indicating that fact, or insert the value from the 3D coordinates obtained in the (n + 1) th step by interpolation such as cubic interpolation. Insert the inserted value.
By classifying the stereo match accuracy of each point according to which stage of the first to n steps the correspondence relationship of each point is determined, and displaying the division for each color, a survey accuracy distribution map can be created.
FIG. 12 is a perspective view of a consumer digital camera used in the digital aerial photography three-dimensional measurement system according to the present invention. FIG. 13 is a plan view of a consumer digital camera used in the digital aerial photography three-dimensional measurement system according to the present invention. FIG. 14 is a front view of a consumer digital camera used in the digital aerial photography three-dimensional measurement system according to the present invention.
The consumer digital camera 14 includes a camera 14a, a controller 14b, an outer frame 14c, an inner frame 14d, a motor 14e, a mounting portion 14f, a coil spring 14g, and the like.
The mounting / control base 15 on which the consumer digital camera 14 is installed is provided with a circular hole 15b in the center and support columns 15a for supporting the consumer digital camera 14 at the four corners.
The camera 14a is installed with the lens facing downward. When fixed to the mounting / control base 15, a photograph can be taken through the hole 15 b. The number of pixels is 12 million or more, and a wide angle lens is attached for use.
The control unit 14b is a box provided on the camera 14a, has a gyro mounted therein, and can detect the inclination with respect to the direction of gravity so that the camera 14a is always directed directly downward.
The aviation GPS antenna 16 and the computer 18 are connected to the control unit 14b via a cable. The camera and the shutter signal transmission cable are connected, and the GPS signal is received and at the same time a signal for releasing the shutter is sent.
The outer frame 14c is a substantially octagonal frame that covers the periphery of the inner frame 14d. The inner frame 14d is connected to the outer frame 14c so as to move freely about both the pitch axis and the roll axis.
The inner frame 14d is a substantially octagonal frame to which the camera 14a and the control unit 14b are attached, and the inclination can be changed by the motor 14e.
The motor 14e is installed on all sides of the camera 14a and operates according to an instruction from the control unit 14b to adjust the tilt of the camera 14a.
The attachment portions 14f are portions that are fixed to the support columns 15a at the four corners of the outer frame 14c. Since the four corners are connected via the coil springs 14g, the camera 14a can be prevented from being shaken by absorbing vibration.
As described above, the digital aerial imaging three-dimensional measurement system according to the present invention includes only a consumer digital camera and a GPS, and is small, light, and low in cost while maintaining accuracy. Aerial photometry will be available for small projects that were difficult to achieve.
In addition, by performing correction or removal of error elements, estimation of the camera line-of-sight direction, etc. with software, it is easy to upgrade to improve accuracy, and calculation can be performed with higher accuracy than with hardware. become.
Furthermore, the consumer digital camera can be replaced, and even if the performance of the consumer digital camera changes, it can be easily handled by adjusting the software, so that it can be operated flexibly.

Since this invention is the above structure, the following effects are acquired. First, the only measuring equipment is a consumer digital camera and GPS, and while maintaining accuracy, it is small, lightweight, and low-cost, so aerial photogrammetry can also be used for small-scale projects that were difficult in terms of cost. become able to.
Second, correction or removal of error elements, estimation of the camera line-of-sight direction, etc. are performed by software, so that version upgrade for improving accuracy is facilitated, and calculation can be performed with higher accuracy than when performed by hardware. become able to.
Thirdly, the consumer digital camera can be replaced, and even if the performance of the consumer digital camera changes, it can be easily handled by adjusting the software, so that it can be operated flexibly.

Claims (5)

Internal orientation elements based on the specifications of the digital camera and a plurality of aerial photographs taken with the digital camera are input to a computer, and the coordinates on the image of the aerial photograph are converted for radial distortion and tangential distortion using the internal orientation elements as coefficients. Geometric correction means for storing the geometrically corrected photograph data in the storage device, and reading the aerial photograph from the storage device and inputting the GPS data received when the aerial photograph was taken into the computer, The difference in the shooting position is calculated based on the altitude information of the GPS data by obtaining the degree of overlap of the two photographs to be performed, and the camera position data associated with each photograph is compared with the latitude and longitude information of the GPS data. Means for continuity analysis stored in a storage device, reading out the geometrically corrected photograph and the camera position from the storage device; Stores tie-point data obtained by image matching to find where multiple points on one photo appear in the other photo when paired with each other in the geometrically corrected photos based on the camera position. Tie point extraction means stored in the apparatus, the internal orientation element, the geometrically corrected photograph, the camera position, and the tie point are read out from a storage device, and the tie point is photographed in a posture in a downward direction at the camera position. Calculate the straight line that reverses the central projection in real space and store the position and orientation data in the storage device with the orientation adjusted so that the twist between the two straight lines in the pair of photos is minimized The orientation means, the internal orientation element, the aerial photograph, and the position and orientation are read from the storage device, and the aerial photograph is geometrically corrected with the internal orientation element. On the other hand, deviation correction means for storing in a storage device the data of a deviation corrected photo re-projected so as to be in a state of being photographed in a posture in a direct downward direction based on the position and orientation, the position and orientation, and the When the deviation-corrected photos are read from the storage device and the overlapping photos of the deviation-corrected photos are paired, the image search is used to find where in the other photo a plurality of points on one photo are located. Stereo matching means for calculating a straight line that reverses the central projection in the real space based on the position and orientation, and storing the three-dimensional coordinate data obtained by the two straight lines in the pair of photographs in a storage device; The internal orientation element, the aerial photograph, the position and orientation, and the three-dimensional coordinates are read from a storage device, and the one represented by the three-dimensional coordinates is displayed in a state having the geometric distortion of the internal orientation elements. Correspondence when shooting in the position / posture orientation is calculated, each point on the aerial photograph is linked to the three-dimensional coordinates, and corrected photo data obtained by converting the image from central projection to orthographic projection is output from the computer A digital aerial three-dimensional measurement system characterized by comprising orthocorrection means. When obtaining pixel values when collating images with photogrammetry tie point extraction or stereo matching, a square shape is used when the point is near the center, and a radial direction when the point is near the outer edge of the image. The frame set in the shape extended to Rotate according to the moving direction of the camera, distorted according to the camera tilt, or scaled according to the camera altitude, and the pixel value acquisition position according to the shape of the frame A pixel value is calculated based on the distance from the center of the surrounding pixels by changing the above. In the camera posture estimation using the coplanar condition and the collinear condition of the photo that overlaps the shooting range in the photogrammetry external orientation, two consecutive photos along the camera traveling direction, the photo and the shooting range A posture estimation method characterized in that estimation is performed with a photograph taken at a position that overlaps and is orthogonal to the traveling direction. When correcting to the central projection in the downward direction by deviation correction of photogrammetry, the center projection is rotated around the lens principal point so that the imaging surface is horizontal with respect to the central projection photo tilted by the camera tilt at the time of photography And re-sampling each point on the imaging surface before rotation onto the imaging surface after rotation, and then re-sampling with the orientation of the sides of each photo aligned. When searching for a point on the other image corresponding to each point on one image by stereo matching of photogrammetry, set the processing accuracy stage and threshold, and use the inductive procedure to determine the 3D coordinates of each point. A stereo matching method characterized by obtaining.

Images