JP2001227945A - Image point coordinating device and method, and recording medium storing image point coordinating program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily specify image points corresponding to plural images with high accuracy. SOLUTION: A first image IM1 and a second image IM2 are displayed in parallel on a monitoring image plane of an image point specifying device. A survey point q1 specifies a first image point Q11 projected to the first image IM1, and an auxiliary line ADL under consideration of the distortion aberration of a projecting optical system in getting both images is overlapped to the second image IM2 and displayed. A second image point Q12 is specified in reference to the auxiliary line ADL. A three-dimensional coordinate of the survey point a1 is determined by allowing the first image point Q11 and the second image point q12 to be corresponding to each other and displayed by dots on a drawing area DRA.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image point associating apparatus for highly accurately associating image points on a subject in each image in photogrammetry for creating a survey map of the object from a plurality of images. The present invention relates to an associating method and a recording medium storing an image point associating program.

[0002]

2. Description of the Related Art Conventionally, in photogrammetry, a stereo image composed of a left-eye image and a right-eye image is obtained using two cameras fixed to each other, and a survey map of a subject is generated based on the stereo image. Is done. Specifically, for example, an image for the left eye and an image for the right eye are displayed in parallel on a monitor screen, and an arbitrary survey point on the subject that is commonly applied to each image is visually viewed by stereoscopic viewing. Survey points on the image, that is, two image points are associated with each other. Then, the three-dimensional coordinates of the survey point are obtained from the associated image point and the distance between the two cameras, and a survey map is generated.

[0003] Image points on a stereo image are associated with each other by specifying an image point with a mouse or the like in one image and searching and specifying an image point corresponding to the specified image point in the other image. In this case, the accuracy of the survey map depends on the ability and experience of the operator, and the association requires a great deal of labor and time. For this reason, when an image point is designated in one image, a visual auxiliary line called an epipolar line is displayed on the other image to facilitate association.

[0004] This epipolar line is formed by a plane including an image point designated by one image and the principal point position of the photographing optical system of both cameras, and the other image (actually, the imaging plane of the camera). Defined as the intersection. For example, in the case of a stereo image, there is no parallax in the vertical direction (Y direction). Therefore, when an image point on one image is designated and the Y coordinate is determined, the epipolar line in the other image matches the Y coordinate. It can be defined as a straight line having coordinates and extending in the horizontal direction (X direction), and the corresponding image point can be regarded as being on the epipolar line.
Therefore, the operator only needs to move the cursor on the epipolar line to associate the parallax in the horizontal direction, thereby greatly improving the work efficiency.

[0005]

However, since the photographing optical system of the camera actually has an inherent distortion, the coordinates of the image points in both images include an error due to the distortion. The position between the epipolar line determined based on the image and the image point on the other image is displaced, which causes a problem that it is not possible to associate image points with high accuracy. Such a problem can be solved by using a photographing optical system without distortion, but a photographing optical system without distortion is very expensive,
Not practical.

The present invention has been made in view of such a point, and even when a photographing optical system having distortion is used, it is possible to easily and accurately associate image points on a plurality of images. It is an object of the present invention to provide an image point associating device and a photogrammetry method using the image point associating device.

[0007]

An image point associating apparatus according to the present invention displays a first image and a second image obtained by imaging the same subject by a photographing optical system having distortion, on a monitor screen. Image display means, a first designating means for designating a first image point where a predetermined survey point on a subject is projected in a first image on a monitor screen, and eliminating an influence of distortion on the first image point Point defining means for defining a plurality of auxiliary points located on a predetermined straight line passing through the first corrected image point obtained by performing the correction and the surveying point, and the second auxiliary point taking into account the distortion Auxiliary image point defining means for defining an auxiliary image point projected on the image, auxiliary image point display means for displaying the auxiliary image point on the second image superimposed on the second image, and auxiliary image point display means for displaying the auxiliary image point on the monitor screen. Of the survey points projected on the second image
It is characterized by comprising a second designating means for designating an image point, and associating means for associating the first image point and the second image point for identifying the relative position of the survey point. This makes it possible to easily and accurately associate image points on a plurality of images even when using an imaging optical system having distortion.

[0008] More specifically, the auxiliary point defining means of the image point associating device further includes a predetermined point which is a predetermined distance away from the first imaging position which is the entrance pupil position of the imaging optical system when the first image is obtained. Is defined by a projection point defining means for defining a projection point projected on an image plane free of distortion, and two-dimensional coordinates of the projection point obtained by the projection point defining means based on Karara's formula. A distortion correction unit that obtains two-dimensional coordinates that take into account the distortion of the projection point by performing the conversion; and that the two-dimensional coordinates that take into account the distortion of the projection point match the two-dimensional coordinates of the first image point. First correction image point specifying means for specifying the relative position of the fixed point and defining the projection point of the specified predetermined point as a first correction image point; and a straight line passing through the first imaging position and the first correction image point as a predetermined straight line The straight line definition means to define and the straight line definition means Auxiliary point extracting means for extracting a plurality of auxiliary points on a defined predetermined straight line and within a predetermined range that can include the survey points, and calculating three-dimensional coordinates of these auxiliary points, respectively. .

The predetermined range of the auxiliary point extracted by the auxiliary point extracting means of the image point associating device is 1 point from the first photographing position.
It is preferably in a range of not less than 50 meters and not more than 50 meters. In consideration of the surveying accuracy, the range of photographing is limited to a range of about 1 to 50 meters from the photographing position. Therefore, the survey points are included in this range with a high probability.

[0010] The auxiliary image point defining means of the image point associating apparatus,
More specifically, based on the three-dimensional coordinates of the auxiliary point defined by the auxiliary point definition means and the three-dimensional coordinates of the second imaging position, which is the entrance pupil position of the imaging optical system when the second image is obtained. A two-dimensional coordinate calculating means for calculating two-dimensional coordinates when the auxiliary point is projected on an image-forming surface free of distortion, and a Karara's two-dimensional coordinate for the auxiliary point obtained by the two-dimensional coordinate calculating means. The image processing apparatus may further include a distortion correction unit configured to calculate a two-dimensional coordinate of the auxiliary image point by performing coordinate conversion in consideration of the distortion based on the equation.

[0011] The image point associating apparatus may include photographing position calculating means for calculating three-dimensional coordinates of the first and second photographing positions, respectively, and the photographing position calculating means is imprinted on the first and second images. The three-dimensional coordinates of the first and second photographing positions are calculated based on the size and shape of the target at the predetermined position and the size and shape of the target image in the first and second images. This makes it unnecessary to measure the reference length one by one when the target is placed near the subject at the time of shooting, and the shooting operation is simplified. More preferably, the photographing position calculating means corrects the target images in the first and second images based on the distortion. As a result, the three-dimensional coordinates of the first and second photographing positions become high-precision values in which distortion has been corrected.

On the monitor screen of the image point associating device,
Preferably, the auxiliary image points are displayed substantially as lines. When the imaging optical system is provided in an electronic still camera having an imaging device in which a predetermined number of pixels are arranged in a grid,
The number of auxiliary points may be any value as long as it is greater than the number of pixels in the horizontal direction of the image sensor, and specifically, 1000 or more is preferable. Thereby, the auxiliary image point to be displayed so as to be superimposed on the second image is displayed as a highly accurate line.

Further, the image point associating method according to the present invention comprises:
A first step of capturing the same subject from the first and second photographing positions by a camera having a photographing optical system having a predetermined distortion to obtain a first image and a second image, and the first and second images In the second step of displaying on the monitor screen, and in the first image displayed on the monitor screen, specifying the first image point on the subject at which the predetermined survey point is projected.
And defining a plurality of auxiliary points on a predetermined straight line passing through the survey point based on the first corrected image point obtained by correcting the first image point specified in the third step based on the distortion. 4th step, a 5th step of defining an auxiliary image point projected on the second image in consideration of the distortion of the auxiliary point, and a 6th step of displaying the auxiliary image point on the monitor screen by superimposing the auxiliary image point on the second image, A seventh image point designating a second image point of the survey point projected on the second image based on the auxiliary image point displayed on the monitor screen
And a step of associating the first image point and the second image point to specify the relative position of the survey point. This makes it possible to easily and accurately associate image points on a plurality of images even when using an imaging optical system having distortion.

The recording medium according to the present invention has an image display routine for displaying, on a monitor screen, a first image and a second image obtained by imaging the same subject by a photographing optical system having a predetermined distortion. Correcting a first image point where a predetermined survey point on a subject specified in the first image displayed on the monitor screen is projected based on the distortion, and calculating the survey point based on the corrected first image point. An auxiliary point defining routine for defining a plurality of auxiliary points on a predetermined straight line passing therethrough; an auxiliary image point defining routine for defining an auxiliary image point projected on the second image by adding the distortion to the auxiliary point; And an auxiliary image point display routine for displaying the image on the monitor screen by superimposing the image on the second image, and projecting the image on the second image specified based on the auxiliary image point displayed on the monitor screen to specify the relative position of the survey point. The second image point of the survey point Stores image point mapping program, characterized in that it comprises a mapping routine for associating the first image point. Thus, the image point association program can be executed on a general-purpose personal computer,
Even when a photographing optical system having distortion is used, it is possible to easily and accurately associate image points on a plurality of images.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image point matching method according to the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram of an image point associating apparatus. The image point associating device includes a CPU 10 for controlling the operation of the entire device, a card reader 12 for reading data relating to an image from an IC memory card 40, and a monitor device 14 for displaying the read image. The image point associating device includes a mouse 16 and a keyboard 17 used for image point designation and various instructions. The CPU 10, the card reader 12, the monitor 14, and the mouse 16 are connected to the bus 30 directly or indirectly.

An input controller 26 connected to a bus 30 is connected to the mouse 16 and the keyboard 17.
6 and the input from the keyboard 17 are transferred to the bus 30, and these input modes and the like are set.

The IC memory card 40 is a card reader 1.
2, a plurality of image data stored in the IC memory card 40 are read into the apparatus as appropriate. The image data is the image data of an image captured by an electronic still camera (not shown).
It includes digital pixel data for a frame, shooting conditions at the time of shooting, data on distortion of the shooting optical system, and data on the size, shape and arrangement of the target imprinted on the image.

The reading operation and the writing operation of the card reader 12 are performed by the recording medium control unit 22 connected to the bus 30.
Is controlled by The hard disk 13 is connected to the recording medium control unit 22, and the hard disk 13
The image data read from the memory card 40 is stored.

A display memory 42 and a working memory 44 are connected to the bus 30, respectively. The display memory 42 holds the contents to be displayed on the monitor device 14 as digital data, and the display control unit 24 connected to the bus 30 stores the held digital data in the analog R for the monitor device 14.
Convert to a GB signal. The work memory 44 is used as a cache memory or the like in the calculation and processing by the CPU 10.

The CPU 10 includes an input state management section 32, a display state management section 34, an arithmetic processing section 36, and a data management section 3.
8 to perform necessary management, calculation, and processing. The input state management unit 32 manages the settings of the mouse 16 and the keyboard 17 and converts input information such as coordinates of the mouse 16 and characters input from the keyboard 17 into predetermined digital data. The display state management unit 34 manages the content to be displayed on the monitor device 14, and changes the display content when there is a change in the display setting or the like. The arithmetic processing unit 36 is used for calculating coordinates of an image point and the like to be described later, correcting distortion, and the like. The data management unit 38 is an IC memory card 40
It manages the contents of data read from the computer, and also manages various coordinate data created on the basis of the read image, data associated with image points, and the like.

A general-purpose personal computer is applied to the image point associating apparatus, and a series of image processing programs for photogrammetry from reading of data relating to an image to drawing are stored in a recording medium detachable from the image point associating apparatus. Is stored in the CD-ROM 60 or the hard disk 13 which is a recording medium built in the image point association apparatus. If the image processing program for photogrammetry is not stored in the hard disk 13, the CPU 10
The image processing program for photogrammetry is read from the CD-ROM 60 via the drive 48 and installed on the hard disk 13. If the CPU 10 has already been installed on the hard disk 13, the CPU 10 reads the image processing program for photogrammetry in the hard disk 13 and executes various processes. Note that an association processing program described later is included in the photogrammetry image processing program.

FIG. 2 is a view conceptually showing a display screen of the monitor device 14. In the image display area IMA of the display screen, a pair of images including the target 50 placed at a predetermined position on the ground surface, that is, a first image IM1 and a second image IM2 are displayed in parallel. First image IM1 and second image I
M2 is an image photographed by a single electronic still camera, and each photographing position is arbitrarily determined by the operator. The first image IM1 and the second image IM2 are displayed based on digital pixel data stored in the IC memory card 40, and W pixels are arranged in the horizontal direction and H pixels are arranged in the vertical direction. W and H are integers (W ≧ H), for example, W = 1000 and H = 800.

A drawing area D is provided below the image display area IMA.
Surveying with RA provided and plotted based on two images
The figure, specifically, a solid line L indicating the outer shape of the road is drawn. Measurement
The quantity map is a horizontal plan view of the road viewed from vertically above. In addition,
Surveying point q₁, Q_Two, Q_Three, Q _FourAnd the target 50
The reference point RP indicating the corner is a point table in the drawing area DRA.
Is shown. A menu MM is provided on the left side of the screen.
Various commands are selectable.

When plotting a survey map, each image of a survey point
The image points at are manually specified and associated
You. For example, in the first image IM1, surveying a corner of a road
Point q ₁First image point AQ₁₁Is manual with mouse 16
When specified, the auxiliary line ADL is superimposed on the second image IM2.
The second image point A is displayed while referring to the auxiliary line ADL.
Q₁₂Is manually specified.

First image point AQ₁₁Is specified by the mouse 16
Pixel coordinates (AX₁₁, AY ₁₁) Get and work
Is stored in the work memory 44. Pixel coordinates (AX₁₁, A
Y₁₁) Are horizontal and vertical from the upper left corner of the first image IM1.
This is the number of pixels in the vertical direction. Second image point AQ₁₂Same for
Like, pixel coordinates (AX₁₂, AY₁₂)
Is obtained.

[0027] Thus the first image point AQ ₁₁ and the second image point AQ ₁₂ as an image point of the surveying point q ₁ are associated with each other, these first image point AQ ₁₁ and the second image point AQ ₁₂ pixels with 3-dimensional coordinates of the survey points q ₁ it is calculated based on the coordinates, surveying point q ₁ is displayed point in the drawing area DRA. Similarly, for the survey points q ₂ , q ₃ , and q ₄ , the image point AQ
₂₁ and image point AQ ₂₂ , image point AQ ₃₁ and image point AQ ₃₂ , image point AQ ₄₁
Image point AQ ₄₂ are associated with each other respectively, the three-dimensional coordinates are calculated with.

The three-dimensional coordinates of the survey points q ₁ , q ₂ , q ₃ , q ₄ are values in a common three-dimensional coordinate system whose origin is the reference point RP of the target 50. Each image point in the first image IM1 and the second image IM2 reference point RP is represented by RP ₁ and RP _2.

In this embodiment, first, the first
Although the image point designation of the image IM1 is performed and the auxiliary line ADL is shown on the second image IM2, the designation order may be reversed, that is, the image point designation of the second image IM2 may be performed first. In this case, the auxiliary line ADL is displayed on the first image IM1.

As described above, when an image point is designated in one image, an auxiliary line ADL which may include an image point corresponding to the other image
Is automatically displayed, so that the operator can easily specify the corresponding image point.

FIG. 3 shows the survey points q _i and 2 of the survey points q _i .
One of the corresponding first image point AQ _i1, a diagram conceptually showing the relation between the second image point AQ _i2. i is a parameter indicating the number of survey points.

When the first image IM1 is obtained, the entrance pupil position of the photographing optical system of the electronic still camera is the first photographing position M1.
The coordinates are represented by three-dimensional coordinates in a reference coordinate system (X, Y, Z) set based on the target 50, that is, reference coordinates (ΔX ₁ , ΔY ₁ , ΔZ ₁ ). The direction of the photographing optical system is indicated by an optical axis CL1, and the rotation angle thereof is the amount of rotational displacement (Δα ₁ , Δα) about the X axis, Y axis, and Z axis.
β ₁ , Δγ ₁ ). These six parameters Δ
X ₁ , ΔY ₁ , ΔZ ₁ , Δα ₁ , Δβ ₁ and Δγ ₁ are called camera parameters.

The camera parameters are read from the first image IM1.
The size of the target 50 by the CPU 10 when
It is calculated based on the normal shape. Similarly, the second image IM2
Camera parameters at the time of obtaining, ie, the second shooting position
M2 (ΔX_Two, ΔY_Two, ΔZ _Two) And the optical axis CL2 (Δ
α_Two, Δβ_Two, Δγ_Two) Is calculated when the image is read.
Calculation of camera parameters is well known in the art.
Here, the description is omitted.

The survey point q _i is a first image IM1 substantially coincident with the imaging plane of the photographing optical system (indicated by a broken line in the figure).
The image is projected as a first image point AQ _i1 (shown by a white circle in the figure). The survey point q _i is ideally the first photographing position M1
Should be located on an extension of a straight line EL connecting the first image point AQ _i1 and the first image point AQ _i1. However, in reality, the photographing optical system has a distortion and deviates from the straight line EL.

Then, the ideal image plane formed by the photographing optical system having no distortion is converted to the first screen S1.
(Indicated by a solid line in the figure), and the image point of the survey point q _i projected on the first screen S1 is set to the first corrected image point Q
_i1 (indicated by a black circle in the figure). The first screen S1 is a plane perpendicular to the optical axis CL1, and the image forming center C1 is located on the optical axis CL1. At this time,
The survey point q _i is located on a straight line EL passing through the first photographing position M1 and the first correction image point Q _i1 .

A first screen coordinate system (X ', Y') having the origin at the image forming center C1 is set on the first screen S1, and the relative position of the first corrected image point Q _i1 on the first screen S1 is determined by the screen. It is represented by coordinates (X _i1 , Y _i1 ). Reference coordinates of the survey points _{q i (qx i, qy i} , qz i) and the relationship between the screen coordinates (X _i1, Y _i1) of the first correction image point Q _i1 (1) and (2) Defined by
In the expressions (1) and (2), j is a parameter indicating the number of images, and in the case of the first image IM1, j = 1.

[0037]

(Equation 1)

[0038]

(Equation 2)

(X _ij , y _ij , z _ij ) shown in the equations (1) and (2) are surveying points q in a camera coordinate system set in an electronic still camera different from the reference coordinate system.
_These are the three-dimensional coordinates of _i . More specifically, a first camera coordinate system (x ′, y ′, z ′) having the first photographing position M1 as an origin is set in the first image IM1, and a survey point q _i in the first camera coordinate system is set. The coordinates are represented by (x _i1 , y _i1 , z _i1 ). Note that the x 'axis is parallel to the X' axis of the first screen S1, and coincides with the horizontal direction of the image sensor provided in the electronic still camera. The y 'axis is parallel to the Y' axis and coincides with the vertical direction of the image sensor. Similarly, a second camera coordinate system (x ″, y ″, z ″) is set, and the coordinates of the survey point q _i in this second camera coordinate system are (x _i2 , y _i2 , z _i2 ).
It is represented by

[0040] (1) The camera coordinates of the survey points _{q i (x ij, y ij} , z ij) and reference coordinates (qx _i, qy _i, q
shows the relationship between z _i), (2) type camera coordinates (x _ij survey points _{q i, y ij, z ij} ) and the screen coordinates (X _i1, Y
_i1 ). Cm in the equation (2) is a parameter determined uniquely for the electronic still camera, and has the same value at both photographing positions M1 and M2.

[0041] Similarly, with respect to the second imaging position M2, the image point of the survey points q _i in the second screen S2 as a second image IM2, and the ideal image plane is a real focal plane, the respective This is shown as two image points AQ _i2 and second corrected image points Q _i2 . Since the survey point q _i is on the straight line EL, the second corrected image point Q _i2 is located on a straight line (hereinafter, referred to as a projection line) PEL obtained by projecting the straight line EL on the second screen S2.

If the projection line PEL is converted in consideration of the distortion, the second image point AQ _i2 should be located on the converted line. In the present embodiment, this converted line is defined as an auxiliary line ADL (shown by a broken line in the figure),
As shown in the figure, the second image IM2 is superimposed and displayed.
As described above, the auxiliary line ADL is determined in consideration of the distortion of the photographing optical system, whereby the second image point AQ _i2 can be easily and accurately specified.

Specifically, n auxiliary points p ₁ to n on a straight line EL and within a predetermined range that can include the survey points q _i.
Extract the p _n, by projecting these auxiliary points p ₁ ~p _n the second screen S2, ₁ further auxiliary image point AP is converted based on the distortion ~AP _n (AP ₁ in FIG. 3 and AP _n Only is shown), and an auxiliary line ADL is generated by connecting successive auxiliary image points AP _{1 to} AP _n . It is preferable that the number n of auxiliary points is large, and is set to a value larger than at least the number W of pixels in the horizontal direction of the first image IM1. In this case, the auxiliary line ADL can be displayed in units of one pixel in the horizontal direction,
A highly accurate and smooth auxiliary line ADL can be obtained.

The predetermined range from which the auxiliary points p _{1 to pn} are extracted is preferably a range from 1 m to 50 m from the first photographing position M 1. The lower limit 1 m and the upper limit 50 m
The shooting range is 1m or more to specify the image point with high accuracy.
It is a numerical value empirically obtained from being limited to about m or less, and is not particularly limited. The predetermined range may be appropriately changed by the menu MM on the screen shown in FIG. In the present embodiment, the number W of pixels in the horizontal direction is 1000, and 1000 auxiliary points p _n (1 ≦ n ≦ 1000) are defined by dividing the range of 1 m to 50 m into 1000 equal parts.

The reference coordinates of auxiliary points _{p n (px n, py n} ,
pz _n ) is calculated by equation (3). (3) t is a parameter indicating the distance in the formula, R is the direction vector indicating the direction of the straight line _{EL (r X, r Y,} r Z).
This direction vector R is a unit vector from the first camera position M1.

[0046]

(Equation 3)

The screen coordinates (X _n2 , Y _n2 ) of the auxiliary point _pn on the second screen S2 are obtained by the above equations (1) and (2). Where the parameter i
And j have n (1 ≦ n ≦ 1000) and 2 respectively.
There is assigned, qx _n, the reference coordinates px _n of qy _n and qz each of the _n auxiliary point p _n, it is py _n and pz _n are substituted.

The auxiliary point p _n of screen coordinates (X _n2,
Y _n2 ) is converted to a value in which the distortion is considered by using the Karara distortion correction model shown in Expression (4).
The transformed point is indicated by the auxiliary image point AP _n, 2 of the converted
The dimensional coordinates are represented by (DX _n2 , DY _n2 ). In the formula (4), D ₂ , D ₄ , D ₆ , P ₁ , P ₂ , Xc and Y
c is a parameter indicating the distortion, especially Xc and Yc are X with respect to the center of the second image IM2 to the optical axis CL ₂ "
Direction and the amount of displacement in the Y ″ direction.

[0049]

(Equation 4)

The two-dimensional coordinates of the auxiliary image point AP _n are (DX _n2 ,
DY _n2 ) is a value in the second screen coordinate system having the origin at the image forming center C2.
It is converted into pixel coordinates (AX _n2 , AY _n2 ) in the image IM2. In equation (5), Px and Py are the horizontal and vertical pixel distances of the image sensor, respectively, and W and H are the numbers of pixels in the horizontal and vertical directions, respectively.

[0051]

(Equation 5)

[0052] Then, AP _n the 1000 auxiliary image point,
Based on the calculated pixel coordinates (AX _n2 , AY _n2 ), they are displayed as, for example, red dots on the second image IM2, and the auxiliary line ADL is obtained by continuously connecting the auxiliary image points AP _n .

With reference to FIGS. 4 to 6, a description will be given of the association processing program executed in the arithmetic processing unit 36 of the CPU 10. FIG.

FIG. 4 is a flowchart showing the main routine of the association processing program. This main routine is started by activating the association processing program. First, in step S102, the first image I
Image data of M1 and the second image IM2 is read from the IC memory card 40. At this time, the photographing conditions at the time of photographing, the distortion aberration data of the photographing optical system, and the data of the size, shape and arrangement of the target 50 are stored in the working memory 44.

Subsequently, step S104 is executed, and the first
Camera parameters (ΔX ₁ , ΔY ₁ , ΔZ) of the image IM1
₁ , Δα ₁ , Δβ ₁ , Δγ ₁ ) and the camera parameters (ΔX ₂ , ΔY ₂ , ΔZ ₂ , Δα ₂ ,
Δβ ₂ , Δγ ₂ ) are calculated. These camera parameters are stored in a predetermined area of the working memory 44. Next, step S106 is executed, and the first image IM1 and the second image IM2 are displayed in the image display area IMA (FIG. 3) set on the screen of the monitor device 14.

Here, in step S108, the value of the parameter i indicating the number of survey points is initialized to 0, and in step S110, the first image IM1 and the second image IM
Whether to continue the correspondence of an image point of the survey points q _i are determined in 2. When continuing, step S112 and subsequent steps are executed, and when not continuing, step S122 is executed.

In step S112, the parameter i is counted up. Subsequently, step S114 is executed, and here, the survey point q _{(i = 1)} in the first image IM1.
Image point, i.e. the first image point AQ ₁₁ is specified manually, the pixel coordinates (AX _11, AY ₁₁₎ is obtained.

Then, the auxiliary line drawing subroutine of step S200 is executed, and the auxiliary line ADL is added to the second image IM2.
Is displayed. Subsequently, in step S116, the survey points q in the second image IM2 are referred to with reference to the auxiliary line ADL.
_{(i = 1)} image point, i.e. the second image point AQ ₁₂ is specified manually, thereby the pixel coordinates (AX _12, AY ₁₂₎ is obtained.

[0059] Subsequently at step S118 is performed, the first image point AQ ₁₁ specified in step S114, and a second image point AQ ₁₂ specified in step S116, the surveying point q
_The pixel coordinates (AX _11, AY ₁₁ ) and (AX _12, AY ₁₂ ) are stored in the working memory 44.

In step S120, the first image point AQ₁₁When
Second image point AQ₁₂Pixel coordinates and camera parameters
Survey point q based on data_{(i = 1)}Reference coordinates (X₁,
Y₁, Z ₁) Is calculated and a point is set in the drawing area DRA (FIG. 3).
Is displayed. The calculation of the reference coordinates is known.
Here, the description is omitted. After the end of step S120,
Returning to step S110, the next survey point q_{(i = 2)}About image point
When specifying, steps S112 to S120 are re-executed.
Is performed.

If it is determined in step S110 that the image point designation is to be ended, the data stored in the working memory 44 in step S122, that is, the first image IM1
And image data of the second image IM2, camera parameters, survey points and data relating to the first and second image points,
After the data related to the survey map is converted into a file based on a predetermined format and stored in the hard disk 13, the main routine, that is, the association processing program ends.

FIG. 5 is a flowchart showing details of the auxiliary line drawing subroutine (step S200) of FIG. First, in step S202, the camera parameters of the first image IM1 and the second image IM2 are
4 is read. Then, the execution direction vector calculation subroutine in step S300, the direction vector R which Teigizukeru the direction of the straight line _{EL (r X, r Y,} r Z) is calculated.

In step S206, step S204
Auxiliary points p _n on the line EL defined is defined by,
Specifically, a numerical value is substituted for the parameter t in the equation (1). In the present embodiment, the distance t is not less than 1 m and not more than 50 m.
A numerical value obtained by dividing the following range into 1000 equal parts is substituted. That is, at step S206, 1000 auxiliary points p
_n (1 ≦ n ≦ 1000) and its reference coordinates are obtained.

[0064] is then step S208 is executed, based on the reference coordinates and the camera parameters of the second image IM2 auxiliary point p _n, screen coordinates of a point obtained through projection is calculated auxiliary point p _n to the second screen S2, Further, the screen coordinates of the 1000 projection points are each subjected to distortion correction. Accordingly, highly precise pixel coordinates of the auxiliary image point AP _n obtained by projecting the auxiliary points p _n in image IM2 is obtained.

Then, in step S210, 100
0 of connecting an auxiliary image point AP _n consecutive auxiliary lines ADL is drawn, a second image on the screen of the monitor device 14 IM2
And the auxiliary line drawing subroutine ends.

FIG. 6 is a flowchart showing details of the direction vector calculation subroutine (step S300) of FIG. Here, a point RB separated by a distance 1 from the first camera position M1, which is the tip of the direction vector R, is referred to as a first image I.
The values of r _X , r _Y, and r _Z are determined so that the distance between the point projected on M1 and the designated first image point AQ _i1 can be minimized.

First, in step S302, appropriate initial values are given to r _X , r _Y , and r _Z of the direction vector R. Next, in step S304, the camera coordinates of the point RB are calculated by equation (1). Then (2)
The screen coordinates of the point RB, the two-dimensional coordinates in consideration of the distortion, and the pixel coordinates are sequentially calculated by the equations (4) and (5) (steps S306, 308, and 31).
0). Then, step S312 is executed, and step S312 is executed.
A table is represented by the square of the distance between the pixel coordinates (AX _RB , AY _RB ) of the point RB obtained in step 310 and the pixel coordinates (AX _i1 , AY _i1 ) of the designated first image point AQ _i1. The calculated error Φ is calculated by equation (6).

[0068]

(Equation 6)

In step S314, it is determined whether or not the error Φ is the minimum value. If it is determined that the error Φ is not the minimum value, the values of r _X , r _Y and r _Z are changed to more approximate values, and step S304 is performed. Return to That is, the values of r _X , r _Y and r _Z are sequentially changed until the error Φ converges to the minimum value, and the calculation of the error Φ is repeated. If it is determined that the error Φ is the minimum value, the current values of r _X , r _Y and r _Z are registered as the direction vector R in step S316, and the direction vector calculation subroutine ends.

[0070]

According to the present invention, image points can be associated with high accuracy on a plurality of images.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of an image point association apparatus according to the present invention.

FIG. 2 is a diagram conceptually showing a display screen of the monitor device shown in FIG.

FIG. 3 is a diagram conceptually showing a relationship between a survey point, a first image point, and a second image point.

FIG. 4 is a flowchart illustrating a main routine of an association processing program.

FIG. 5 is a flowchart showing details of an auxiliary line drawing subroutine of FIG. 4;

FIG. 6 is a flowchart showing details of a direction vector calculation subroutine of FIG. 5;

[Explanation of symbols]

10 CPU 14 monitor device 16 mouse 60 CD-ROM IM1 first image IM2 second image _{q 1, q 2, q 3} , q 4 surveying point _{AQ 11, AQ 21, AQ 31} , AQ 41 first image point AQ _12, AQ _22, AQ _32, AQ ₄₂ second image point p ₁ ~p _n auxiliary points AP ₁ ~AP _n auxiliary image point ADL auxiliary line

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] November 29, 2000 (200.11.
29)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0049

[Correction method] Change

[Correction contents]

[0049]

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0051

[Correction method] Change

[Correction contents]

[0051]

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA04 BB28 EE08 FF05 FF26 FF61 JJ03 JJ26 QQ00 QQ03 QQ23 QQ24 QQ28 2F112 AC06 BA06 CA08 CA12 FA03 FA07 FA21 5B057 CA12 CA16 CC01 CD12 DA07 DB02 DC09

Claims (11)

[Claims] 1. An image display means for displaying, on a monitor screen, a first image and a second image obtained by forming an image of the same object by a photographing optical system having distortion, and the first image on the monitor screen. A first image point designating a first image point on the subject at which a predetermined survey point is projected
Specifying means for correcting the first image point to eliminate the influence of the distortion, and obtaining a plurality of auxiliary points located on a predetermined straight line passing through the obtained first corrected image point and the surveying point. Auxiliary point defining means for defining, auxiliary image point defining means for defining an auxiliary image point projected on the second image in consideration of the distortion, and superimposing the auxiliary image point on the second image. Auxiliary image point display means for displaying on the monitor screen, based on the auxiliary image points displayed on the monitor screen,
Second specifying means for specifying a second image point of the survey point projected on the second image, and associating the first image point and the second image point to specify a relative position of the survey point An image point associating device, comprising: associating means. 2. The method according to claim 1, wherein the auxiliary point defining unit includes a predetermined point that is a predetermined distance away from a first imaging position that is an entrance pupil position of the imaging optical system when the first image is obtained.
A projection point defining means for defining a projection point projected on the image plane having no distortion, and coordinates of the two-dimensional coordinates of the projection point obtained by the projection point defining means based on Karara's formula. A distortion correcting unit that obtains two-dimensional coordinates of the projection point by taking into account the distortion, by performing the conversion; and that the two-dimensional coordinates of the projection point taking into account the distortion are two-dimensional coordinates of the first image point. The relative position of the predetermined point is specified so as to match the projection point of the specified point,
First correction image point specifying means for defining a correction image point; straight line definition means for defining a straight line passing through the first shooting position and the first correction image point as the predetermined straight line; Auxiliary point extracting means for extracting a plurality of the auxiliary points on the predetermined straight line and within a predetermined range that may include the survey points, and calculating three-dimensional coordinates of these auxiliary points, respectively. The image point associating apparatus according to claim 1, wherein: 3. The image point associating apparatus according to claim 2, wherein the predetermined range is a range of not less than 1 meter and not more than 50 meters from the first photographing position. 4. An auxiliary image point defining means comprising: three-dimensional coordinates of the auxiliary point defined by the auxiliary point defining means; and an entrance pupil position of the imaging optical system when the second image is obtained. Two-dimensional coordinate calculating means for calculating two-dimensional coordinates when the auxiliary point is projected on an image plane free of distortion based on three-dimensional coordinates of a certain second imaging position; and the two-dimensional coordinate calculating means. 2 of the auxiliary points obtained by
And a distortion correction unit that calculates two-dimensional coordinates of the auxiliary image point by performing coordinate conversion on the dimensional coordinates in consideration of the distortion based on Karara's formula. 2. The image point associating apparatus according to claim 1. 5. The first and second images based on the size and shape of a target at a predetermined position projected on the first and second images and the size and shape of the target image in the first and second images. And a photographing position calculating means for calculating three-dimensional coordinates of first and second photographing positions, which are entrance pupil positions of the photographing optical system when the second image is obtained. The image point associating device according to the above. 6. The image point associating apparatus according to claim 5, wherein the photographing position calculating means corrects the target image in the first and second images based on the distortion. 7. The image point associating apparatus according to claim 1, wherein the auxiliary image points are displayed substantially as lines on the monitor screen. 8. An electronic still camera having an imaging device in which a predetermined number of pixels are arranged in a grid pattern, wherein the number of the auxiliary points is larger than the number of pixels of the imaging device in a horizontal direction. The image point associating apparatus according to claim 1, wherein: 9. The number of the auxiliary points is at least 1000.
The image point associating apparatus according to claim 1, wherein the number is one. 10. A first step of capturing an image of the same subject from a first and a second imaging position by a camera having an imaging optical system having a predetermined distortion, and obtaining a first image and a second image; A second step of displaying the first and second images on a monitor screen; and a step of designating, in the first image displayed on the monitor screen, a first image point where a predetermined survey point on the subject is projected. 3 steps: correcting the first image point designated in the third step based on the distortion, and providing a plurality of auxiliary points on a predetermined straight line passing through the survey point based on the obtained first corrected image point. A fourth step of defining a point; a fifth step of defining an auxiliary image point where the auxiliary point is projected on the second image in consideration of the distortion; and superimposing the auxiliary image point on the second image. The first to be displayed on the monitor screen Steps and, on the basis of the auxiliary image point displayed on the monitor screen,
A seventh step of designating a second image point of the survey point projected on the second image, and associating the first image point and the second image point to specify a relative position of the survey point An image point associating method, comprising: an eighth step. 11. An image display routine for displaying, on a monitor screen, a first image and a second image obtained by forming an image of the same subject by a photographing optical system having a predetermined distortion, and displaying the image on the monitor screen. Correcting a first image point where a predetermined survey point on the subject specified in the specified first image is projected based on the distortion, and correcting the corrected first image point.
An auxiliary point definition routine for defining a plurality of auxiliary points on a predetermined straight line passing through the survey point based on an image point; and an auxiliary image point which is obtained by projecting the auxiliary point on the second image in consideration of the distortion. An auxiliary image point definition routine for defining, an auxiliary image point display routine for displaying the auxiliary image point on the monitor screen by superimposing the auxiliary image point on the monitor screen, and specifying the relative position of the survey point on the monitor screen. A second image point of the surveying point specified based on the displayed auxiliary image point and projected on the second image, and an association routine for associating the second image point with the first image point. A recording medium storing an image point association program.