WO2008152740A1

WO2008152740A1 - Digital aerial photographing three-dimensional measurement system

Info

Publication number: WO2008152740A1
Application number: PCT/JP2007/062373
Authority: WO
Inventors: Fumio Shinohara
Original assignee: Information & Science Techno-System Co., Ltd.
Priority date: 2007-06-13
Filing date: 2007-06-13
Publication date: 2008-12-18
Also published as: JPWO2008152740A1

Abstract

A digital aerial photographing three-dimensional measurement system capable of calculating a three-dimensional coordinate with high accuracy from an image captured by means of a digital camera is provided. The digital aerial photographing three-dimensional measurement system is characterized in that the internal orientation element based on the specification of a digital camera, a plurality of aerial photographs captured by the digital camera, and the GPS data received with the photographing of the aerial photographs are inputted to a computer and that corrected photographs are outputted from the computer by a geometric correction means, a continuous analysis means, a tie-point extraction means, an external orientation means, a deviation correction means, stereo matching means, and an orthographic correction means.

Description

Description Digital Sky 3D Measurement System Technical Field

The present invention is a system related to photogrammetry performed from an aircraft using a digital camera as an imaging device. Background art

Photogrammetry is a technique that calculates the three-dimensional coordinates of an object based on the information on the position and orientation of the force camera when the object is photographed from multiple positions and angles. The Among photogrammetry, aerial photogrammetry is the one that calculates the latitude / longitude / altitude of a feature or terrain by photographing the direction directly below the flying aircraft, especially a chemical film as a photographic recording medium. So <digital aerial photogrammetry is what uses electronic media

Normally, the image recorded on the imaging surface of the force camera is the one that is projected by central projection within the range that falls within the angle of view of the camera. In other words, when a straight line that passes through the main point of the lens from each point on the imaging surface is extended, the first line that hits the straight line is recorded at that point.

Therefore, the three-dimensional coordinates of each point on a photo can be reversed if the optical system of the force camera that took the photo and the position and orientation of the imaging surface at the time of taking the photo are known. Somewhere on each straight line you get when you follow. At this time

If there is another photo of the same object taken at different positions and orientations, a straight line can be obtained for each point in the same way. The straight lines that can be obtained for the same point are always different and always intersect. In other words, at the intersection, the coordinates to be found.

Conventional navigational photogrammetry often uses a large-scale system that has been developed exclusively for its use, with high accuracy, high cost, high implementation costs, and high operational costs. However, it is difficult to do so unless it is a project. The market is growing rapidly, and there are cameras with high resolution that can withstand photogrammetry.

As described in Japanese Patent Application Laid-Open No. 2000-107-1089, which is Patent Document 1.

The position specifying unit specifies the position based on the amount of error comparing the image data captured by the image pickup unit and the geographical data corresponding to the shooting range of the image pickup unit at each point.

In addition, an invention that can perform positioning even under conditions in which the number of GPS satellite supplements is not sufficient is also disclosed.

As described in Japanese Patent Application Laid-Open No. 2 0 06-0 7 9 5 21, which is Patent Document 2.

3D coordinate information at a predetermined position of the shooting object using multiple image information taken with a digital camera from multiple points with different shooting positions for the shooting object and 3D coordinate information of the shooting object An invention is also disclosed in which image coordinate information on image information to be obtained is obtained, and an RGB value corresponding to the obtained image coordinate information is obtained to form an orthographic projection image of the object to be photographed.

As described in Japanese Patent Application Laid-Open No. 2 0 0 1 — 1 1 4 0 4 7, which is Patent Document 3, an image from an arbitrary viewpoint is synthesized using an image from a camera installed in a car. In addition, an invention that presents the driver with an easy-to-determine object other than the road surface by synthesizing the road surface projection image with an index for distinguishing objects other than the road surface by the road surface index combining means is also disclosed. ing.

As described in Japanese Patent Application Laid-Open No. 2000-0615, which is Patent Document 4, an image input means for acquiring two stereo images whose horizontal axes are parallel projections, and image input An image matching means for calculating the similarity between the corrected image obtained by deforming the local region of one of the two images obtained by the means and the other image; and image matching An invention comprising a 3D point cloud generation means for generating a 3D point cloud based on the similarity calculated by the means is also disclosed.

As described in Table 2 of Japanese Patent Application Laid-Open No. 2000-042086, a reference image and a reference image are associated with each other based on a predetermined correlation. By setting multiple windows, and by expanding and contracting the multiple windows formed in the reference image, the distance between adjacent windows in the reference image can be reduced. Or cancel the overlap, and the reference image window and reference image An invention that performs stereo matching with the image window and restores the shape of the object has also been made public.

However, since the conventional dedicated system is entirely composed of hardware, the entire device tends to be large and maintenance is troublesome. Moreover, even if the technology improves, it is difficult to exchange partly.

Note that the invention described in Japanese Patent Application Laid-Open No. 2 0 0 7 — 1 0 8 0 2 9, which is Patent Document 1, uses a sky image camera to capture the entire periphery with infrared rays, and is therefore dedicated. This hardware is necessary and cannot be realized with a consumer digital camera.

The invention described in Japanese Patent Laid-Open No. 2 0 06-0 7 9 5 21, which is Patent Document 2, forms an orthographic projection image from three-dimensional coordinate information of an object to be imaged. It does not calculate three-dimensional coordinates by shooting an object from multiple points with a digital camera.

The invention described in Japanese Patent Application Laid-Open No. 2 0 0 1 — 1 1 4 0 4 7, which is Patent Document 3, displays an image of the surroundings of the vehicle in an easy-to-understand manner by combining images taken with a camera mounted on the vehicle. It is not intended to calculate high-precision 3D coordinates by correcting the image.

The invention described in Japanese Patent Application Laid-Open No. 2 0 0 6-1 9 5 7 5 8, which is Patent Document 4, is a method in which two cameras are fixed, photographed while moving in parallel, and stereo-matched. Even if the position and posture of the camera are indefinite, it cannot be handled.

The invention described in Japanese Patent Laid-Open No. 2 0 0 4-0 3 8 6 60, which is Patent Document 5, divides a standard image obtained by photographing the same object and a reference image into the same number of windows, respectively. The stereo matching is based on the correspondence between the windows, and it does not calculate the three-dimensional coordinates with high accuracy by finding the correspondence for each point over the entire area where the two images overlap.

In view of the above, an object of the present invention is to provide a digital aerial three-dimensional measurement system capable of calculating three-dimensional coordinates with high accuracy from an image photographed with a digital camera. Disclosure of the invention

In order to solve the above problems, the present invention is based on the specifications of a digital camera. Geometric elements and multiple aerial photographs taken with the digital camera are input to a computer, and the coordinates on the image of the aerial photograph are transformed for radial and tangential distortions using the internal orientation elements as coefficients. A correction means for storing the corrected photograph data in the storage device, and reading the aerial photograph from the storage device and inputting the GPS data received when the aerial photograph was taken to the computer. Calculate the difference in the shooting position based on the altitude information of the GPS data by determining the overlapping degree of two consecutive photos, and compare the latitude and longitude information of the GPS data with the power associated with each photo Continuity analysis means for storing camera position data in a storage device, the geometrically corrected photograph and the camera position are read from the storage device, and the geometry is based on the camera position. When pairing adjacent photos of the correct photos, the data of the typo した found by image matching where multiple points on one photo appear in the other photo The tie point extracting means stored in the storage device, the internal orientation element, the geometrically corrected photograph, the camera position, and the tie point are read from the storage device, and the tie point is read from the camera. Assuming that the camera was photographed in the position directly below at the position L, the straight line that follows the central projection in real space was calculated, and the attitude between the two lines in the pair of photographs was minimized. Means for storing the position and orientation data adjusted in the storage device in the storage device, reading the internal orientation element, the aerial photograph and the position and orientation from the storage device, and geometrically correcting the aerial photograph with the internal orientation element Deviation correction means for storing in a storage device the data of a deviation-corrected photograph that has been reprojected so as to be in a state of being photographed in a downward direction based on the position and orientation; And the deviation-corrected photos are read from the storage device, and the images that overlap each other in the deviation-corrected photos are paired, and a plurality of points on one photo are the other photo. The image is searched for by image comparison, and a straight line that reverses the central projection in real space is calculated based on the position and orientation, and is obtained from the two straight lines in the pair of photographs. Stereo matching means for storing data of three-dimensional coordinates in a storage device, the internal orientation element, the aerial photograph, the position and orientation, and the three-dimensional coordinates are read from the storage device and expressed by the three-dimensional coordinates. A number of internal orientation elements By calculating the correspondence when shooting with the above-mentioned position and orientation with some distortion, linking each point on the aerial photograph to the three-dimensional coordinates, the image is orthogonally projected from the central projection The composition of the digital aerial 3D measurement system, which is composed of an orthorectification means that outputs from the computer the data of the corrected photograph converted to, and the typographic point extraction or stepping of the photogrammetry In the pixel value acquisition when matching images by rhe matching, when the point is near the center, it has a square shape, and when the point is near the outer edge of the image, the shape is elongated in the radial direction. The set frame is rotated according to the direction of camera movement, distorted according to the camera tilt, or scaled according to the camera altitude, and the pixel value acquisition position is adjusted according to the frame shape. The image matching method is characterized in that the pixel value is calculated based on the distance from the center of the surrounding pixel, and the coplanar conditions and the coplanar conditions for the overlapping of the shooting range in the photogrammetry external orientation. Use line conditions In the camera posture estimation performed in this way, there are two consecutive photographs along the camera traveling direction, and a photograph taken at a position where the photograph and the photographing range overlap and are orthogonal to the traveling direction. At the time of capturing a central projection that is tilted by the camera tilt at the time of taking a picture when correcting to the central projection in the downward direction by correcting the deviation of photogrammetry and the posture estimation method characterized by estimation Rotate around the lens principal point so that the imaging surface is horizontal, reproject each point on the imaging surface before rotation onto the rotated imaging surface, and then align the sides of each photo In the case of finding a point on the other image corresponding to each point on one image by stereo matching of photogrammetry, and a method of correcting accuracy characterized by re-sampling Set thresholds and recursively That you determine the three-dimensional coordinates of it was realized Ri by the and the stereo matching method which is characterized. Brief Description of Drawings

Fig. 1 is a flowchart showing the flow of processing of the digital aerial 3D measurement system of the present invention. Fig. 2 shows the processing of the digital aerial 3D measurement system of the present invention, and the relationship between data and equipment. Fig. 3 shows the configuration of the digital aerial 3D measurement system computer of the present invention, and Fig. 4 shows the deformation of the figure in the image correlation method of the digital aerial 3D measurement system of the present invention. Fig. 5 is a diagram showing the deformation of the figure in the image correlation method of the digital aerial photography 3D measurement system according to the present invention, and Fig. 6 is the pixel value of the digital aerial photography 3D measurement system according to the present invention. Fig. 7 shows acquisition, Fig. 7 shows coplanar conditions and collinear conditions, and Fig. 8 shows digital aerial photography according to the present invention. Fig. 9 shows a set of images for estimating the position and orientation of the 3D measurement system. Fig. 9 shows a method for correcting the deviation of the digital aerial 3D measurement system of the present invention. Fig. 10 shows the present invention. Fig. 11 shows a sample of a digital aerial 3D measurement system with corrected displacement. Fig. 1 1 shows the stereo matching method of the digital aerial 3D measurement system of the present invention. Fig. 2 is a perspective view of a consumer digital camera used in the digital aerial 3D measurement system of the present invention, and Fig. 1 3 is a consumer digital camera used in the digital aerial 3D measurement system of the present invention. Fig. 14 is a front view of a consumer digital camera used in the digital aerial 3D measurement system of the present invention. Good form for carrying out the invention

Hereinafter, the digital photographing 3D measurement system according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a flowchart showing the flow of processing of the digital aerial photography 3D measurement system according to the present invention. FIG. 2 which is a chart is a diagram showing the relationship between the processing, data and apparatus of the digital aerial three-dimensional measurement system according to the present invention. FIG. 3 is a diagram showing a configuration of a digital aerial imaging three-dimensional measurement system computer according to the present invention.

As shown in Fig. 1, the digital aerial 3D measurement system 1 consists of geometric correction, 2 linkage analysis 3, type point extraction 4, external orientation 5, displacement correction 6, stereo Vching

7 and orthorectification 8 means.

-What correction 2 uses the parameters obtained in internal orientation 9 to generate a photograph that corrects the distortion included in each photograph.

In continuity analysis 3, each photo is shot in sync with the GPS signal and shot, but it is a single photo that serves as a reference for determining which photo specifically corresponds to which GPS record. When the corresponding GPS record is input, the remaining image and the GPS record are collated and analyzed based on that.

In the point extraction 4, data for external orientation 5 is generated. Specifically, the characteristic points that are shared by using image matching technology between images with a part of the shooting range overlapped. Find and save the coordinates on the image as data In the external orientation 5, the position of the camera's imaging surface at the time of capturing each photo is determined with high accuracy based on the position data measured by the GPS and the photos taken by the camera. presume.

Specifically, the initial condition is the assumption that each photograph was taken from the position as recorded in the GPS, and it can be obtained by following the optical system from typpoint 4a. Calculate the position and orientation of the imaging surface using a solution such as the Newton method so that the twist of each ray that should originally be on the same plane is minimized.

If the features (ground control points) that have already been separated into geographical coordinates are included in the photo, the information can be used for this estimation, but this is not essential.

In the displacement correction 6, in order to simplify the calculation of the stereo matching 7, a re-projected photograph is generated to the image that should have been taken when the photograph was taken in the direct downward direction. Specifically, using the results of external orientation 5 for each photo, the coordinates on the image of each point on the photo are converted to the positions that should be projected when taken in the direct downward direction. Reoma

Resample by aligning the orientation of the photos to be vchinged 5

In the stereo matching 7, of the multiple photos after the internal orientation 9 and external orientation 5 have been completed and the displacement corrected 6, two of the large overlapping areas appear in the 3 Calculate dimension information.

Specifically, using the image matching technology, the correspondence between the points on each photo (determining the force at which the point on one side appears on the other side, Deriving the intersection of straight lines obtained by tracing back the system and finding the coordinates Since the photograph is a raster image, this processing is done by finding the correspondence for each pixel. .

Since the photo is a central projection, when a deep (high altitude) object appears in the vicinity of the outer edge of the photo, an area that cannot be seen in the shadow (falling down) occurs. Because it varies with the angle, it differs from photo to photo, and you cannot perform stereo matching7 on this range. Therefore, if you want to get 3D information, you can make up for the fall in each photo 5, which requires photos from various positions or angles.

Orthorectification 8 reprojects the central projection into an orthographic projection. The photo is a central projection As a result of the stereo matching 7, the coordinates of each point on the photo are obtained, and after mapping each point to this coordinate, the depth axis (altitude axis) is crushed so that it is only the vertical and horizontal axes. Then, an orthographically projected photo is obtained.In addition, 3D data is generated based on the coordinates obtained as a result of stereo-yching7, and the orthorectified photo8 is used as the texture. If used, a 3D photographic map can be obtained.

As shown in Fig. 2, the geometric correction 2 means inputs the data of the internal orientation element 9b and the aerial photograph 10a, and outputs the data of the geometrically corrected photograph 2a. Using the value of the internal orientation element 9 b as a coefficient, a radial distortion equation and a tangential distortion equation are determined, and the inverse transformation is applied to the aerial photograph 10 a. Specifically, it is a process that transforms the coordinates on the image by inverse transformation of the distortion and resamples it into a raster image by an internal method such as cubic interpolation.

The data of the internal orientation element 9 b is the internal orientation with T data of the force meter specification 9 a as input.

It is created by the process of 9. Among the data of camera specification 9a, it is a parameter that specifies the optical system of consumer digital camera 14 and the parameter that represents the distortion of the optical system determined by the process of internal orientation 9

The data for aerial photograph 10 a is taken a; It is an electronic image file with a shell IJ on the output format of a consumer digital camera 14. Each photo is taken so that 60% or more of the + 曰 -shadow range overlaps the previous and next photos, and at least one of the flight routes is a non-consecutive flight route (ie Taken from a crossing or parallel path) and 30% or more of each other's shooting range overlap each other.

Geometrically corrected photo 2a data is an electronic image file obtained by removing the distortion inherent in the optical system of consumer digital camera 14 from each of aerial photo 10a data. The data in 9a is data that has attributes such as the principal force lens of the consumer digital force meter 14 and the lens, that is, principal point position, focal length 画素 pixel size, pixel spacing, etc. Is a set of

The work of internal orientation 9 is the distortion of the optical system of the consumer digital camera 14, that is, the ideal center projection determined by the camera specification 9 a and the actual center projection Measure the error between and and determine the parameter to correct it. The central projection in photography is ideally determined by the camera's principal point position, focal length, pixel dimensions, and pixel spacing, but the decision itself must use approximate conditions. In addition, due to problems with machining accuracy, the actual lens is not ideal, it is necessary to measure distortion aberration etc. in combination and determine the parameters to correct the error from the ideal central projection .

Specifically, for photographs of multiple objects with known absolute coordinates taken from a position where the absolute coordinates are known, the coordinates on the image where these objects should appear under ideal conditions ( And the coordinates on the image in which these objects are actually reflected are fit into the equation for the coordinate transformation on the image representing the radial distortion and tangential distortion. The coefficient is determined when this equation corresponds to the equation that reproduces the distortion of the optical system. The photographs taken with the optical system are inversely transformed by using the on-image coordinate transformation formula determined by applying the coefficients determined in this way to the distortion formulas. Can be converted to images taken under dynamic conditions

For the aerial photography 10 work, the consumer digital camera 14 and the aviation GPS receiver 16 are installed on the aircraft via the mounting and control base 15 and the functions of the aviation GPS receiver 16 are used. Using the functions of the mounting and control pedestal 15 to send a shutter signal synchronized with the recording of the aviation GPS receiver 16 to the consumer digital camera 14 Use the functions of the control pedestal 15 to take a photo from the aircraft with the function of the consumer digital camera 14 while keeping the consumer digital camera 14 in a position facing directly downward. .

Consumer digital cameras 14 are commercially available single-lens reflex digital cameras and the like. It has an interface that accepts a shutter signal from the outside, and attaches it to the control base 1 5. The mounting part is versatile and can be replaced when a higher-performance camera appears.

Mounting and control pedestal 15 is a pedestal mounted on an aircraft, equipped with a consumer digital force meter 14 and an aviation GPS receiver 16. A machine that absorbs vibrations from aircraft and tilts the mounting part with a motor so that the consumer digital camera 14 always faces directly below. Has the ability. It also has a function to receive a signal from the aviation GPS receiver 16 and record it in the built-in memory force, and at the same time send a digital camera for consumer use 14 in sync with the signal reception. Have.

The aviation GPS receiver 16 is a receiver for three-dimensional positioning of an aircraft based on the arrival time of radio waves of time signals emitted from satellites. The electronic board is installed. Installed in the control pedestal 15, attach only the antenna to an appropriate place outside the aircraft or inside the aircraft where radio waves can be easily received, and wire the antenna and board with a cable.

As shown in Fig. 2, the means of continuity analysis 3 inputs aerial photograph 10a and GPS data 12a data, and outputs camera position 3a data. Entering a standard photo of aerial photos 10 a and the GPS data 1 2 a corresponding to that photo, the photos are first linked to the GPS record. After that, one photo from the aerial photograph 10 0a that is not linked to the GPS record is taken from the photo associated with the GPS record. Select several points on one photo, find out where they are on the other photo by image matching using the image correlation method, and determine the overlap of the two images. Based on this overlap and the altitude information of the GPS record linked to one of the photos, the difference between the shooting positions is converted into an actual distance, and compared with the latitude and longitude information of GPS data 1 2 a, Connect the record of GPS data 1 2 a that should correspond to the geographic coordinates of the other photo. This is recursively repeated, and the GPS record corresponding to each photograph of the aerial photograph 1 O a is taken out from the GPS data 1 2 a and connected to obtain camera position 3 a.

The GPS data 1 2 a data is data obtained by improving the accuracy of the aerial GPS data 10 b in which the flight path was recorded at the time of aerial photography 10 by performing the GPS analysis 1 2. G P S Includes data such as time, geographic coordinates, and moving speed.

The data at camera position 3a is the GPS data 1 2a for each of the aerial photos 1 0a, and records (GPS time, geographic coordinates, movement speed, etc.) corresponding to the moment of shooting are recorded. It is the linked data.

The GPS analysis 1 2 processing is more accurate by performing RTK-GPS analysis using aerial GPS data 10 b as mobile station data and fixed-point GPS data 1 1 a as base station data. Calculate flight path of aerial photography 1 0. The aerial GPS data 10 b data is the data recorded by the aerial GPS receiver 16 when the flight route during the aerial photography 10 is implemented. Includes data such as GPS time, geographic coordinates, travel speed, and carrier waves.

The fixed-point GPS data 1 1 a data is the data observed during the same period including or including the fixed-point GPS receiver 1 7 installed at the point where the geographical coordinates are known in advance, and The geographical coordinates of the point that were known in advance. Includes data such as G PS time and carrier wave.

Fixed point positioning 1 1 work is close to the flight path of aerial photography 10 (preferably the entire flight path is within a radius of 10 km), and the fixed point GPS is received at a point where the geographical coordinates are known in advance. Aircraft 17 will be installed, and positioning will be performed during the same period as aerial photography 10 or during that period. If an electronic reference point managed by the Geospatial Information Authority of Japan exists near the flight path of aerial photography 10, observations using that electronic reference point satisfy the requirements for fixed-point positioning 1 1, and from that electronic reference point, The provided data can be used as fixed-point GPS data 1 1 a.

The fixed point GPS receiver 17 is a receiver that is installed at a location where the geographical coordinates are known in advance, and records the carrier wave at the geographical coordinates based on the arrival time of the radio wave of the time signal emitted from the satellite.

As shown in Fig. 2, the tie point extraction 4 means inputs the data of the geometrically corrected photograph 2a and the force mela position 3a, and outputs the data of the tie point 4a. It is also possible to input the internal orientation element 9b. Based on the relationship between the camera position 3a and the position information, the geometrically corrected photographs 2a that overlap each other are paired. The determination of whether or not they overlap is determined by whether or not the distance between the camera positions 3a corresponding to each photo exceeds a threshold given from the outside. When the internal orientation element 9 b is input, this threshold value is calculated by estimating the size of the shooting range from the angle of view of the internal orientation element 9 b and the altitude of the camera position 3 a. Next, for each pair, select several points on one photo, find out where they are on the other photo by image matching using the image correlation method, and determine the coordinates of each image. Record as correspondence.

The data for typpoint 4a is adjacent to the data for geometrically corrected data 2a. This is a record of the coordinates of each point in the image for a pair of matching photos.

As shown in Fig. 2, the method of external orientation 5 is to input the data of geometrically corrected photo 2a, camera position 3a, typpoint 4a and internal orientation element 9b, and the data of position and orientation 5a. Output. In addition, data of G CP data 1 3 b can be input. With respect to the coordinates on the image of tie point 4a, the center projection is traced back using the internal orientation element 9b, and the incident ray on the imaging surface is calculated. The incident light beam to the corresponding point on the other photograph should be on the same plane. Therefore, by using a solution such as Newton's method, the position and orientation of each imaging surface when the twist of each corresponding ray is minimized are calculated, and the position of the imaging surface at the time of each photography is calculated. Use posture information.

In addition, when GCP data 1 3 b is used, the incident ray is similarly calculated for each point on the photograph corresponding to the point of GCP data 1 3 b, and the geography associated with that point is calculated using the above solution. Information on the position and orientation of each imaging plane is calculated so that the distance between the coordinates and the ray is also the shortest at the same time. In this case, since the coordinates on the image of GCP data 1 3 b need to be deformed according to geometrically corrected photo 2 a, the same preprocessing as geometric correction 2 is performed using internal orientation element 9 b. Apply to 1 3 b.

The data of the position / orientation 5a is data of the position and orientation of the camera imaging surface at the moment when each photograph of the geometrically corrected photograph 2a was taken.

The data of G C P data 1 3 b is the data that combines the coordinates on the image and the geographical coordinates of the photo that includes the point of G C P information 1 3 a in the aerial photo 10 a.

The processing of GCP designation 1 3 is to find the point corresponding to the location on the aerial photograph 1 0 a image from the state of the ground reference point of GCP information 1 3 a, and to determine the geographical coordinate (latitude of the ground reference point) , Longitude, altitude) and the image coordinates of the image.

GCP information 1 3 a data is recorded in aerial photography 1 0 a in aerial photography 1 0 and is easy to interpret on aerial photography 1 0 a in aerial photography 1 0 (Scenery photo, etc., which is a clue to image interpretation) and geographic coordinate data at that point. As shown in Fig. 2, the displacement correction 6 means inputs the data of the internal orientation element 9b, the aerial photograph 10a and the position and orientation 5a, and outputs the data of the displacement corrected photograph 6a. From the position and orientation of position 5a, re-project the photo of geometrically corrected photo 2a so that it is in the shooting state when the camera is facing directly below.

Note that the geometrically corrected photograph 2 a is an image that has been resampled once, so re-processing it may result in loss of image definition. Perform geometric correction 2 using, and recalculate from the point. In other words, the processing here converts the coordinates on the image by the inverse transformation of the distortion, cancels the position / posture inclination, converts the coordinates on the image in the downward direction, Resampled into a raster image by the method.

The data of the deviation corrected photo 6a is an electronic image file obtained by projecting each of the photos of the geometrically corrected photo 2a into a photo when taken in the downward direction. Actually, in order to avoid the loss of detail in the image, instead of directly processing the geometrically corrected photo 2a, it is generated by reprocessing from the aerial photo 10a where the inherent distortion of the optical system is removed. To do.

As shown in FIG. 2, the stereo matching 7 means inputs the data of the internal orientation element 9 b, the displacement corrected photo 6 a and the position and orientation 5 a and outputs the data of the three-dimensional coordinate 7 a. A pair of photos that have been corrected is selected based on the position information of position and orientation 5a. With regard to the shooting range that is commonly included in this pair of photos, the image correlation method is used to find where each point on one photo appears on the other photo. Then, for each point on each photo, a straight line that reverses the central projection from the real space is calculated based on the position and orientation of the position and orientation 5a. Since the data of the position and orientation 5a is obtained through the processing of the external orientation 5, the straight line extending from a certain point in one photo is 0 or the distance from the straight line extending from the same point in the other photo is 0 or Very short. Therefore, the midpoint of the distance is the geographical coordinate of that point. The correspondence between the coordinates on the image and the geographical coordinates obtained in this way is recorded.

The data of the three-dimensional coordinate 7 a is data that associates the coordinates on the image and the geographical coordinates of the point that is common to the other photos for each of the photos of the displacement corrected photo 6 a. As shown in Fig. 2, the orthorectification 8 means inputs the data of the internal orientation element 9b, the aerial photograph 10a and the three-dimensional coordinate 7a, and outputs the corrected photograph 8a. When the terrain represented by the geographical coordinates of the 3D coordinate 7a is captured with the position and orientation 5a in the state with the geometric distortion of the internal orientation element 9b, The projection from the point is calculated as the correspondence between image coordinates and geographic coordinates. Based on this correspondence, geographic coordinates are connected to each point on the image of the aerial photograph 10a, and based on the geographic coordinates, each image coordinate is converted from a central projection to an orthogonal projection. Record the re-sampled coordinates on the converted image into a raster image by an interpolation method such as cubic interpolation.

The data of the corrected photo 8a is an electronic image file that is an electronic image file that is obtained by projecting the range shown in each of the aerial photos 10a and the other and im into an orthogonal projection. Each means of the imaging 3D measurement system 1 is realized by a computer 18. As shown in FIG. 3, the computer 18 includes an input device 18 a, a central processing device 18 b, a storage device 18 c, and an output device 18 d.

The input device 1 8 a is a device that takes in the input data 1 8 e from the outside into the computer 1 8. Inputs 7 to 1 8 e are stored in accordance with instructions from the central processing unit 1 8 b.

Stored in 1 8 c.

Input data 1 8 e is internal orientation element 9 b, aerial photograph 10 0 a, aerial GPS data

1 2 a, G C P data 1 3 b, etc. It may be provided in a state of being recorded on a recording medium.

1 8 b So central processing unit is a command execution and control by analyzing instructions

. If necessary, retrieve data from pLiLi 'device 18 c or save data to storage device 18 c

The storage device 18 c is a device that records programs, input data 18 e, internal data 18 f, output data 18 g, and the like, and may be constructed as a database.

The internal data 1 8 f is data created in the computer 1 8 during the process. Geometrically corrected photo 2a, camera position 3a, typpoint 4a, position There are data such as 5a, displacement corrected photo 6a, 3D coordinate 7a, etc.

The output device 18 d is a device for taking out output data 18 g from the computer 18 to the outside. The output data 18 g is called from the storage device 18 c by the instruction of the central processing unit 18 b.

The output data 1 8 g is data such as the corrected photo 8 a. It is output by means such as displaying on a screen, printing on paper, or recording on a recording medium. FIG. 4 and FIG. 5 are diagrams showing the deformation of a figure in the image correlation method of the digital aerial photography three-dimensional measurement system according to the present invention. FIG. 6 is a diagram showing the acquisition of pixel values of the digital aerial imaging three-dimensional measurement system according to the present invention.

Image verification is a technology that detects where two or more digital images appear on one image but on the other. In particular, in photogrammetry, for each point on each photo recorded as a raster image, which point on the other photo corresponds to that point is detected using the image correlation method. In this method, this method is used by means of typpoint extraction 4 and stereo matching 7.

Assuming that two digital images are image A and image B. First, for the original image A, select one point A that is the target of image verification. The area around the point A is surrounded by a square frame A with, and the pixel value A of each point A contained in the frame A is acquired.

Next, a frame B of the same size is placed on the other image B, the pixel value B of the point B included therein is acquired, and the correlation coefficient with the pixel value A in the frame A acquired previously is obtained. . While fixing frame A on image A, move frame B on the other image B little by little to find the correlation number one after another. When the correlation coefficient is the highest, center point B of frame B Is considered to be a point that coincides with the center point A of frame A on image A.

As shown in FIG. 4, in the present invention, the shape of the frame 20 a placed on the image 19 is deformed.

. Even if they are the same, the point 2 1 shown near the outer edge of image 1 9 is a dot in the center.

Compared to 20, it falls down and the surrounding features are recorded on the photograph in a shape that is stretched radially. Therefore, if the position is near the center of the image 9, it is a square frame 20a, and if it is near the outer edge, it is a square frame 21a. The

In tie-point extraction 4, the displacement correction 6 is not performed and the orientation of each image is not uniform, so as shown in the upper part of FIG. Rotate frame 20 a to align.

Next, as shown in the middle of Fig. 5, the frame 20 a is distorted according to the inclination of the camera in the direction directly below. The tilt information used at this time can be input from the outside, but if it is not input, it is treated as having no tilt. In addition, as shown in the lower part of Fig. 5, the frame 20 a is enlarged or reduced according to the camera height. The altitude information used at this time can be obtained from the camera position 3a.

Also, as shown in Fig. 6, as the frame 20a is deformed, the position for obtaining the pixel value 20b for calculating the correlation coefficient is also changed. In the square frame 20 0 a without deformation, it is assumed that the pixel value 2 0 b was obtained from the center of each pixel, and the position of each center point is changed according to the deformation of the frame 20 0 a.

When acquiring the pixel value 2 0 b from the changed acquisition position, first, if the acquisition position coincides with any pixel center, the pixel value 2 0 b of that pixel is used as it is.

Next, when the acquisition position is on a straight line connecting the centers of two consecutive pixels, the values obtained by interpolating the pixel values 20 b of those pixels by the distance between the center and the acquisition position are used.

In other cases, the values obtained by interpolating the pixel values 2 0 b of the nearest four pixels surrounding the acquisition position by the distance between the center and the acquisition position are used. The equation used for interpolation follows an interpolation method such as cubic interpolation.

In order to determine the geographical coordinates of each point on the photo, information on the camera's position and orientation when each photo was taken is required. Based on the actual measurement data, the position and orientation of the imaging device at the time of each photo shoot can be obtained to some extent accurately. This is used as the initial value to estimate the position and posture at the time of taking each photo. In the present invention, this method is used in the external orientation 5 method.

The upper part of Fig. 7 shows the coplanar conditions. Note that the coplanar condition refers to the point on the imaging surface corresponding to each point of two photographs of the same point, respectively. These lens principal points have the property of being on the same plane.

In the conventional method, the point 2 3 a on the imaging surface 2 3 corresponding to the point 2 2 a on the imaging surface 2 2 is searched for by using the image correlation method for two photographs whose imaging ranges overlap each other. The geographical coordinates of point 2 2 a and point 2 3 a can be expressed by an expression using the camera position and orientation as variables. At this time, depending on the coplanar conditions, point 2 2 a, the lens principal point 2 2 b, point 2 3 a, and lens principal point 2 3 b are on the same plane. If you find many pairs of points that correspond between two photos, it is possible to estimate the error included in the position and orientation and eliminate the error by combining these equations and conditions. it can.

The lower part of Fig. 7 shows the collinear conditions. Note that the collinear condition is a property that the point of the imaging range, the point on the imaging surface corresponding to the point, and the lens principal point exist on the same straight line.

The estimation method based on coplanar conditions can remove variations in position and orientation information between images, but if the entire image contains the same error, it cannot be removed. In particular, the error derived from the degree of freedom of rotation about the straight line connecting the lens principal points of the two photographs to which the coplanar condition is applied may remain large. Therefore, in the conventional method, the point where the geographical coordinates that can be seen in any one of the photographs is known as the ground reference point, and recorded on the ground reference point 2 4 c in the shooting range 2 4 b and on the imaging surface 24. Using the collinear conditions for the selected point 2 4 a and the lens principal point 2 4 d, the error contained in the position and attitude information of the photograph is estimated, and the error is removed.

In the present invention, by using three or more photographs, an estimation accuracy equivalent to that of the conventional method is realized from only the measured position data. In other words, only the position data derived by R T K-GPS is used without using the measured data of the attitude and the ground reference point.

FIG. 8 is a diagram showing a set of images in position and posture estimation of the digital aerial imaging three-dimensional measurement system according to the present invention. In aerial photography 10, images 25 b taken at positions orthogonal to these traveling directions are overlapped so that the two continuous images 25, 25 a along the traveling direction overlap the shooting range. prepare.

As shown in the upper diagram of Fig. 8, first, while taking a picture while flying on a straight line to obtain images 25 and 25a, flight path 25 c Thus, image 25b is obtained. Or, as shown in the lower figure of Fig. 8, first take a picture while flying on a straight line to obtain images 25 and 25a, then fly in the direction perpendicular to the straight line. The image 25 b is obtained by the flight path 25 d to be photographed.

The coordinates on the image of tie point 4a obtained by applying tie point extraction 4 to two pairs of images 2 5, 2 5 a 2 5 b obtained in this way. Then, using the internal orientation element 9 b to reverse the central projection and calculate the incident light on the imaging surface at the time of each photo shoot, the incident light on the point on one photo of each pair And the incident light at the corresponding point on the other photo are straight lines connecting the points on the imaging surface and the lens principal points at the same point, and depending on the coplanar conditions, Should be on the same plane. Therefore, assuming that each of these images was taken at the position of the position information of camera position 3a as an initial condition and the camera was taken with the camera facing directly downward, a solution such as the Newton method was used. Using this, the position and orientation of each imaging plane are calculated so that the twisting of the corresponding rays between the three pairs is minimized, and this is equal to the position and orientation of the imaging plane at the time of each photo shoot. . At this time, since the lens principal points of the three photographs are not on the same straight line, the degrees of freedom of rotation around the straight line connecting the lens principal points for these three coplanar conditions cancel each other out. At the same time, the error resulting from this freedom is negligible. Therefore, the camera position and orientation information can be estimated with very high accuracy without using the collinear condition. In addition to these three images, if there is an image that overlaps with any of these three images, applying the coplanar condition to that image also adjusts the position and orientation information of that image. At the same time, the position and orientation information of the original three images can be estimated with high accuracy. Similarly, the number of images to be estimated can be increased inductively. That is, when a group of three or more images satisfies the conditions for estimation by this method, an image not included in those groups overlaps with any image included in those groups. If so, the image can also be subject to estimation by this method in addition to the group. If a point of GCP data 1 3 b is shown in any of these images, the center projection is reversed using the internal orientation element 9 b to calculate the incident ray at that point, and Using the solution, the twist between each ray is minimized and at the same time the distance between the ray and the geographic coordinates associated with that point In addition, the position and orientation information of each imaging surface is calculated so as to be as short as possible. In this case, since it is necessary to transform the coordinates on the image of the GCP data 1 3 b to match the geometrically corrected photograph 2 a, the same preprocessing as in the geometric correction 2 is performed using the internal orientation element 9 b. Apply to data 1 3 b.

In the present invention, prior to stereo matching 7, displacement correction 6 is applied to each photograph. In other words, the central projection photograph tilted by the camera tilt at the time of taking a picture is re-projected to the central projection in the direct downward direction.

FIG. 9 is a diagram showing a deviation correction method of the digital aerial imaging three-dimensional measurement system according to the present invention. At low latitudes, the latitudes and meridians locally form square squares, but all of the photos taken with the camera tilted 26 have distorted squares. Using the information of the slope of the image, the image is corrected so that it becomes a square cell, and a displacement corrected photo 2 7 is generated.

First, the imaging surface 26 a tilted with respect to the imaging range 26 b is rotated around the lens principal point 26 c so that the light rays from the imaging range 26 b remain horizontal. Next, for each point on the image before rotation, the position of each point is moved to a position where each corresponding light beam intersects the rotated imaging surface 27 a.

FIG. 10 is a diagram showing a resample of an image subjected to displacement correction in the digital aerial photography three-dimensional measurement system according to the present invention. In order to create a raster image for each point moved by leveling the photographed picture 28, resample the image and record it as a displacement-corrected image. At this time, in order to simplify the subsequent calculation of stereo matching 7, resampling is performed so that the directions of the edges of the images to be stereo matched are aligned. In particular, in the case of a series of photographs, the image is rotated 29 a so that the sides of the resampled image are aligned with the aircraft traveling direction 28 a at the time of each photographing.

For two or more digital images, based on image matching, find the coordinates on the image where one image appears on one image and the coordinates on the image where the same image appears on the other image. By taking into account the camera's position and orientation, the 3D coordinates of the camera itself can be obtained. In the present invention, this method is used in the method of stereo matching 7.

For two images with a wide range shown in common, the position and orientation estimation method is used. Obtain the position and orientation of the camera when they were photographed. Then, for each point on one image, a corresponding point is found from the other image by the image correlation method.

FIG. 11 is a diagram showing a stereo-matching method of the digital aerial imaging three-dimensional measurement system according to the present invention. The optical system is turned upside down from the point 30 0a on one imaging surface 30 and the straight line 30d that extends to the shooting range 30c through the lens principal point 30b is uniquely determined. You can. Similarly, the equation of the straight line 3 1 d extending from the corresponding point 3 1 a on the other imaging surface 3 1 to the imaging range 3 1 c through the main lens point 3 1 b is reversed. Can also be determined uniquely.

Point 3 0 a and point 3 1 a are the same object point projected by different central projection systems, so line 3 0 d and line 3 1 d should intersect, and these lines 3 0 The result of solving the equations of d and 3 1 d simultaneously corresponds to the coordinate of the intersection, that is, the three-dimensional coordinate 3 2. If the straight lines 3 0 d and 3 1 d do not intersect, find the midpoint of the two straight lines and use it as the 3D coordinate 3 2.

In the present invention, for two images A and B that are overlapped with each other with the displacement correction 6, a point on image B corresponding to each point 30 0a on image A in an inductive procedure. Find 3 1 a and find the 3D coordinates 3 2 of each point.

Step 0

The number of stages of processing accuracy is n, and threshold values t h (1) to t h (n) and threshold values d i f f (l) to d i f f (n) are determined. Place moving point X a on image A and moving point X b on image B, and determine the initial position and the direction to move them.

Step i (i = 1 to n)

(1st step)

Of the points on image A, the moving point X is the point closest to the initial position with respect to the direction in which the moving point is moved among the points that have not yet been determined in steps 0 to (i 1 1). Put a. Also, the moving point X b is placed at the initial position of image B.

(2nd step)

The frames centered at the moving point Xa and the moving point Xb are defined as Wa and Wb, respectively, and the correlation coefficient cc between the frame Wa and the frame Wb is obtained by the image correlation method. The correlation coefficient cc is the threshold th ( i) If so, go to the third step, otherwise go to the fourth step.

(3rd step)

Take the small range S a including the point X a, calculate the correlation coefficient between the frame W s centered on that point and the frame W b for each point included in the small range S a, Let X a 'be the high point. If the distance between the moving point Xa and the point Xa 'on the image is less than or equal to the threshold value d i f f (i), go to the fifth step, otherwise go to the fourth step.

(4th step)

Move the moving point X b to the next point in image B and return to the second step. If there is no next point, proceed to Step 6.

(5th step)

Let X b be the point corresponding to moving point X a

(6th step)

Move moving point X a to the next point on image A and return to the second step. If there is no next point, go to step (i + 1).

Step (n + 1)

Obtain the 3D coordinates for the points for which the correspondence was determined in steps 0 to n.

Step (n + 2)

For points for which the correspondence relationship has not been determined in steps 0 to n, enter a value to that effect, or use 3D interpolation from the 3D coordinates obtained in step (n + 1). Enter the value entered by the method.

Depending on which step in steps 1 to n the correspondence of each point is determined, the stereo matching accuracy of that point is classified, and when that category is displayed by color, a survey accuracy distribution map can be created. You can

FIG. 12 is a perspective view of a consumer digital camera used in the digital aerial photography three-dimensional measurement system according to the present invention. FIG. 13 is a plan view of a consumer digital camera used in the digital aerial photography 3D measurement system of the present invention. FIG. 14 is a front view of a consumer digital camera used in the digital aerial photography 3D measurement system of the present invention. The consumer digital camera 14 has a camera 14a, a controller 14b, an outer frame 14c, an inner frame 14d, a motor 14e, a mounting 14f, and It consists of 14 g.

In addition, the mounting and control base 15 for installing the consumer digital camera 14 has a circular hole 15b in the center, and supports 1 for supporting the consumer digital camera 1 4 at the four corners. 5 a is provided.

The camera 14a is installed with the lens facing down. Installation. When fixed to the control pedestal 15, the photograph can be taken through the hole 15 b. The number of pixels is 12,000,000 or more, and a wide-angle lens is attached for use.

The control unit 14 b is a box provided on the camera 14 a, has a gyro inside, detects the inclination with respect to the direction of gravity, and the camera 14 a is always directly below. It can be controlled to turn in the direction.

The control unit 14 b is connected with an aeronautical GPS antenna 16, a computer 18, and the like via a cable. Connect the camera and the shutter signal transmission cable, receive the GPS signal, and send a signal to turn off the shutter at the same time.

The outer frame 14c is a substantially octagonal frame that covers the periphery of the inner frame 14d. The inner frame 14 d is connected to the outer frame 14 c in such a way that it freely moves on both the pitch axis and the roll axis.

The inner frame 14 d is a substantially octagonal frame to which the camera 14 a and the control unit 14 b are attached. The inclination of the inner frame 14 d can be changed by the motor 14 e.

The motor 14 e is installed in the four directions of the camera 14 a and operates according to instructions from the control unit 14 b to adjust the inclination of the camera 14 a.

The attachment portions 14 f are portions fixed to the support columns 15 a at the four corners of the outer frame 14 c. Since it is connected to the four corners via a coil spring 14 g, the vibration can be absorbed and the camera 14 a can be prevented from shaking.

As described above, the digital aerial 3D measurement system according to the present invention has only a digital camera for consumer use and a GPS as measuring instruments, and is small, light, and low in cost while maintaining accuracy. As a result, aerial photometry can be used even for small-scale projects that were difficult in terms of cost. In addition, by performing correction or removal of error elements, estimating the camera's line-of-sight direction, etc. with software, it is easy to upgrade the version to improve accuracy. It will be possible to calculate with higher accuracy.

In addition, consumer digital cameras can be replaced, and even if the performance of consumer digital cameras changes, it can be easily accommodated by simply adjusting the software. It is possible to do this. Industrial applicability

Since the present invention has the above configuration, the following effects can be obtained. First, the measurement equipment is only a consumer digital camera and GPS, and while maintaining accuracy, it is small, lightweight, and low cost. Aerial photogrammetry will also be available.

Secondly, the software can be used to correct or remove error elements, estimate the direction of the camera's line of sight, etc. Compared to the case, it becomes possible to calculate with high accuracy.

Third, consumer digital cameras can be replaced, and even if the performance of consumer digital cameras changes, it can be easily accommodated by adjusting the software. It can be used.

Claims

Claims An internal orientation element based on the specifications of the digital camera and a plurality of aerial photographs taken with the digital camera are input to a computer, and the radial orientation and tangential distortion are calculated using the internal orientation element as a coefficient. Geometric correction means for storing in the storage device the data of the geometrically corrected photo obtained by converting the coordinates on the image of the aerial photo; and the GPS data received when the aerial photo is read out from the storage device and taken Is input to the computer, the difference between two consecutive photographs of the aerial photograph is obtained, the difference in the shooting position is calculated based on the altitude information of the GPS data, and compared with the latitude and longitude information of the GPS data. A continuity analysis means for storing camera position data associated with each photograph in a storage device, and reading the geometrically corrected photograph and the camera position from the storage device. Based on the camera position, when the adjacent photos of the geometrically corrected photos are taken as a pair, the image collation shows where the multiple points on one photo appear in the other photo. Tie-point extracting means for storing the tie-point data found in the storage device in a storage device, reading out the internal orientation element, the geometrically corrected photograph, the camera position, and the tie-point from the storage device; Assuming that the camera was photographed at the camera position with a posture in the direct downward direction, a straight line that reverses the central projection in the real space is calculated, and the blur between the two straight lines in the pair of photos is minimized. Means for external orientation for storing data of position and orientation adjusted in posture in a storage device; reading the internal orientation element, the aerial photograph and the position and orientation from the storage device; Deviation that stores in a storage device the data of a deviation-corrected photo that has been re-projected so as to be photographed in a posture in a direct downward direction based on the position and orientation of the one that has been geometrically corrected by the internal orientation element When the correction means, the position and orientation, and the displacement-corrected photo are read from the storage device, and the overlapped photos of the displacement-corrected photos are paired, a plurality of points on one photo are Find the location of the other photo by image matching, calculate a straight line that reverses the central projection in real space based on the position and orientation, and use the two straight lines in the pair of photos. Sought 3D , Stereo-matching means for storing coordinate data in a storage device, the internal orientation element, the aerial photograph, the position and orientation, and the three-dimensional coordinates are read from the storage device, and represented by the three-dimensional coordinates. To calculate the corresponding relationship when the object is photographed in the position and orientation with the geometric distortion of the internal orientation element, linking each point on the aerial photograph to the three-dimensional coordinate, A digital aerial 3D measurement system characterized by comprising orthorectification means that outputs from a computer the data of a corrected photograph converted from a central projection to an orthographic projection.

When obtaining pixel values when collating images with photogrammetry typographical points or stereo matching, when the point is near the center, it is a square shape, and when the point is near the outer edge of the image The frame set with the shape extended in the radial direction is rotated according to the camera traveling direction, is distorted according to the camera tilt, or is scaled according to the camera altitude, to the shape of the frame. At the same time, the pixel value is calculated based on the distance from the surrounding pixel center by changing the acquisition position of the pixel value.

In the camera posture estimation using the coplanar condition and collinear condition of the photos with overlapping shooting ranges in the photogrammetry external orientation, two consecutive photos along the camera traveling direction, and A posture estimation method characterized in that estimation is performed based on a photograph and a photograph taken at a position where the photographing range overlaps and is orthogonal to the traveling direction. When correcting to the central projection in the downward direction by deviation correction of photogrammetry, the imaging surface should be horizontal with respect to the photo of the central projection tilted by the tilt of the force lens when taking a photo. Rotating around the lens principal point, reprojecting each point on the imaging surface before rotation onto the imaging surface after rotation, and then resampling with the orientation of the sides of each photo aligned Deviation correction method.

In photogrammetry stereo matching, when finding a point on the other image corresponding to each point on one image, set the processing accuracy level and threshold, and use an inductive procedure to set each point. A stereo matching method characterized by obtaining three-dimensional coordinates.