JP7493793B2 - Image orientation method, image orientation device, image orientation system, and image orientation program - Google Patents

Description

The present invention relates to an image orientation method, an image orientation device, an image orientation system, and an image orientation program.

Cameras installed outdoors, such as on rivers, roads, and parks, can change the shooting direction and magnification by remote control, and the shooting angle of view is often not fixed. When measuring the size and moving speed of an object captured in an image taken by such a camera, it is necessary to obtain the interior and exterior orientation parameters of the camera when the image was taken.

Conventional image orientation methods involve placing markers that are easily visible in captured images at multiple locations within the area to be orientated. Then, image orientation must be performed after measuring the position coordinates of the markers in a coordinate system fixed on the ground (see, for example, Patent Document 1).

Relisted 2005/017644

However, in some cases, such as on flooded rivers or on roads in use, the installation of markers may be prohibited or restricted. Therefore, with conventional image location methods that require the installation of markers within the location area, location work may be prohibited or restricted.

The object of the present invention is to provide a location method that does not require the installation of markers within the location target area.

One invention for achieving the above object is an image orientation method including the steps of photographing an object with a first camera whose position is fixed relative to the object and acquiring a first image, photographing at least a portion of the object and acquiring a plurality of second images, identifying corresponding points for each of a plurality of points on the object from the first image and at least one of the plurality of second images, and calculating orientation elements of the first camera at the time of capturing the first image based on the position coordinates of the corresponding points for each of the plurality of points. Other features of the invention will be made clear by the description in this specification.

The present invention provides a location method that does not require the installation of markers within the location target area.

FIG. 1 illustrates an example of an image orientation system. FIG. 2 is a diagram showing an example of an arrangement of a first camera and a second camera. FIG. 2 is a diagram showing an example of a first image. FIG. 13 is a diagram showing an example of a plurality of second images. FIG. 2 is a diagram for explaining various coordinate systems and collinearity conditions. FIG. 2 is a diagram showing an example of a functional block realized in the image localization device. FIG. 13 is a diagram showing an example of a plurality of feature points identified from a first image. FIG. 11 is a diagram showing an example of a plurality of feature points identified from a second image. FIG. 11 is a diagram showing an example of a plurality of corresponding points identified from a first image. FIG. 13 is a diagram showing an example of a plurality of corresponding points identified from a second image. FIG. 2 is a diagram illustrating an example of a three-dimensional image. 1 is a flowchart showing an example of a process executed by an image location device; 11 is a flowchart showing a modified example of the process executed by the image location device. FIG. 2 is a diagram showing an example of an arrangement of a first camera and a plurality of third cameras.

First Embodiment
<Image Orientation System>
Fig. 1 is a diagram showing the configuration of an image orientation system 1 according to the present embodiment. The image orientation system 1 includes a first camera C1, a second camera C2, and an image orientation device 2. Fig. 2 is a diagram explaining the arrangement of the first camera C1 and the second camera C2.

[First Camera]
The first camera C1 is a camera for obtaining a first image F1 (details will be described later). The first camera C1 in this embodiment is a camera whose position is fixed with respect to the subject. The subject in this embodiment includes rivers, roads, bridges, buildings, etc. Therefore, the first camera C1 can also be said to be a camera whose position is fixed with respect to the ground. The first camera C1 has variable attitude, lens focal length, shooting angle of view, etc., which can be controlled, for example, by remote control. The first camera C1 is, for example, a surveillance camera installed for the purpose of disaster prevention, crime prevention, etc.

The first camera C1 may be a portable video camera, a digital camera, a camera mounted on a mobile device, etc. In this case, these devices may be fixed to the ground using a tripod or the like.

[Second Camera]
The second camera C2 is a camera for obtaining a plurality of second images F2i (i is an integer, details will be described later). The second camera C2 in this embodiment is a camera whose position is movable relative to the subject. The second camera C2 is, for example, a camera mounted on a moving body such as an aircraft, a vehicle, or a ship. The second camera C2 may be installed on these moving bodies, or may be supported by the hands or shoulders of crew members of these moving bodies.

In this embodiment, the second camera C2 is a camera installed on the aircraft A. FIG. 2 shows the flight route R of the aircraft A. While the aircraft A is flying, the second camera C2 photographs the subject, thereby obtaining a second image F2i. When the second camera C2 is loaded onto the aircraft, the direction in which the subject is photographed is not particularly limited, but it is preferable to photograph the subject from a direction that is angled downward vertically. This is preferable because it allows images to be obtained that show not only the sides of the subject that are perpendicular to the vertical direction, but also the sides that are parallel to the vertical direction, thereby obtaining three-dimensional image data, which will be described later, with high accuracy.

The second camera C2 may be a camera for obtaining still images or a camera for obtaining moving images. If the second camera C2 is a camera for obtaining still images, it obtains the second image F2i by photographing a subject while the aircraft A is flying at a predetermined position or at a predetermined time interval. If the second camera C2 is a camera for obtaining moving images, one frame of the moving image at a predetermined time may be used as the second image F2i.

In the following, when there is no need to distinguish between the first camera C1 and the second camera C2, they will simply be referred to as "cameras."

[Hardware configuration of image orientation device]
1 shows an example of a hardware configuration of the image localization device 2. The image localization device 2 is a computer including a CPU (Central Processing Unit) 20, a main memory device 21, an auxiliary memory device 22, an input device 23, an output device 24, and a communication device 25.

The CPU 20 reads and executes the programs stored in the main memory 21 to control the operation of each of the above-mentioned devices of the image orientation device 2. The CPU 20 also performs arithmetic operations and logical operations according to the programs.

The main memory device 21 is a storage device for temporarily storing programs and data being executed by the CPU 20. The main memory device 21 can be configured, for example, from a semiconductor storage device such as a RAM (Random Access Memory).

The auxiliary storage device 22 is a storage device for storing programs and various data executed or processed by the CPU 20. The auxiliary storage device 22 can be configured, for example, from a magnetic storage device such as a hard disk, an optical storage device such as a CD or DVD, or a semiconductor storage device such as a flash memory. The auxiliary storage device 22 stores an image orientation program and image data composed of various codes according to this embodiment.

The image orientation program is a program for implementing the various functions of the image orientation device 2 (described in detail below).

The image data is data for displaying each of a plurality of images, including a first image F1 and a plurality of second images F2i, on a display device, which is one of the output devices 24. FIG. 3A is a diagram showing an example of a first image F1. FIG. 3B is a diagram showing an example of a plurality of second images F2i. In this example, eight second images F2i (i = 1 to 8) are shown.

The first image F1 is an image acquired by photographing a subject with the first camera C1. The first image F1 is an image that is the subject of image orientation by the image orientation system 1 of this embodiment. In other words, the image orientation system 1 of this embodiment calculates the orientation elements at the time when the first image F1 was captured by the first camera C1.

Here, the orientation parameters include exterior orientation parameters and interior orientation parameters. Exterior orientation parameters are the position and orientation of the camera at the time of capturing an image. Interior orientation parameters are the geometric characteristics of the camera, such as the focal length of the camera lens and the lens distortion coefficient. In this specification, when the term "orientation parameters" is used simply, it means both exterior orientation parameters and interior orientation parameters.

Each of the multiple second images F2i is an image captured and acquired by the second camera C2. Also, each of the multiple second images F2i is an image captured and acquired by the second camera C2 of at least a portion of the subject. As described above, the second camera C2 is a camera installed on an aircraft, vehicle, etc., and therefore can capture images of the subject from multiple different positions while the aircraft, vehicle, etc. is moving. This allows multiple second images F2i to be acquired.

The multiple second images F2i are images from which the image orientation system 1 obtains information necessary for performing image orientation of the first image F1. Any one point on the subject captured in the first image F1 must be captured in at least one of the multiple second images F2i.

The multiple second images F2i are also images used by a three-dimensional image data creation unit 32 (described later) to create three-dimensional image data of the subject. The three-dimensional image data creation unit 32 can have any configuration. When the three-dimensional image data creation unit 32 creates three-dimensional image data, any one point on the subject needs to be captured in at least two of the multiple second images F2i.

In the following, when there is no need to distinguish between the first image F1 and the second image F2i, they will simply be referred to as "images."

The input device 23 is a user interface for importing various data from outside the image localization device 2 into the image localization device 2. The input device 23 is a device such as a keyboard, a mouse, and a microphone.

The output device 24 is a user interface for outputting the processing results, etc., inside the image orientation device 2 to the outside of the image orientation device 2. The output device 24 is a device such as a display device, such as an LCD (Liquid Crystal Display), a printer, a speaker, etc.

The communication device 25 is a network interface for connecting the image orientation device 2 to various communication networks. The communication device 25 receives data from other computers via a network such as the Internet or a LAN (Local Area Network), and stores the received data in the auxiliary storage device 22 or the main storage device 21. The communication device 25 also transmits data stored in the auxiliary storage device 22 or the main storage device 21 to other computers via the network. The communication device 25 is, for example, a device such as a network card.

[Various coordinate systems and collinearity conditions]
4 is a diagram for explaining various coordinate systems and collinearity conditions. Before explaining the functional blocks of the image orientation device 2, for the convenience of the following explanation, the absolute coordinate system, the photograph coordinate system, and the camera coordinate system are defined, and the relationship between them is explained. Furthermore, the collinearity conditions are explained.

The absolute coordinate system is a three-dimensional Cartesian coordinate system that has an origin at a point on the ground and coordinate axes in three directions relative to the ground. The position coordinates of the camera position O in the absolute coordinate system are defined as point O ( _X0 , _Y0 , _Z0 ). The position coordinates of a point P on the subject in the absolute coordinate system are defined as point P (X, Y, Z).

The photograph coordinate system is a two-dimensional Cartesian coordinate system that has an origin at a point on the image and has coordinate axes in two directions relative to the image. The position coordinates in the photograph coordinate system that correspond to point P (X, Y, Z) are defined as point p (x, y).

The camera coordinate system is a three-dimensional Cartesian coordinate system that has its origin at the camera position and has coordinate axes in three directions relative to the camera. The camera position coordinate o in the camera coordinate system is defined as point o(0,0,0). Furthermore, the position coordinate p in the camera coordinate system that corresponds to point P(X,Y,Z) is defined as point p(x,y,-c) using coordinate values in the photograph coordinate system. c is the focal length of the camera lens.

Point O, point p, and point P are aligned on a straight line. This is called the collinearity condition, and by formulating the collinearity condition, the following equation is obtained:

Here, k is a positive real number. Furthermore, the matrix R is a matrix for converting from the camera coordinate system to the absolute coordinate system, and is an orthogonal matrix a _ij (i=1 to 3, j=1 to 3) with 3 rows and 3 columns. In other words, the transposed matrix of the matrix R is equal to the inverse matrix of the matrix R. Furthermore, when the matrix R is determined, the attitude of the camera, which is one of the exterior orientation parameters, is specified. In other words, each component of the matrix R is a parameter resulting from the exterior orientation parameters.

By eliminating k from equation 1, the following two equations are obtained.

Here, when errors Δx and Δy occur in the position coordinates of the photograph coordinate system (left side of Equations 2-1 and 2-2) due to internal orientation parameters such as lens distortion, Equations (2-1) and (2-2) are rewritten as Equations 3-1 and 3-2 below, respectively.

In other words, the interior orientation parameters are modeled as errors Δx and Δy. Several models have been proposed for expressing the errors Δx and Δy using the interior orientation parameters, but any of them may be used, and there are no particular limitations.

In this specification, "calculating the orientation parameters" means to obtain the numerical values of the orientation parameters, or to obtain the numerical values of parameters, etc., resulting from the orientation parameters. In addition, in this specification, "calculating the exterior orientation parameters of the camera" means to obtain the position coordinates in the absolute coordinate system of the camera and the parameters that specify the camera's attitude. The "parameters that specify the camera's attitude" may be the components of the matrix R (there are three independent parameters), or the rotation angles ω, φ, and κ of the camera coordinates relative to the X-axis, Y-axis, and Z-axis of the absolute coordinate system, respectively. In addition, in this specification, "calculating the interior orientation parameters of the camera" means to obtain the interior orientation parameters included in the model of the focal length c of the camera lens and the errors Δx and Δy resulting from the interior orientation parameters of the camera.

[Functional block of image orientation device]
5 is a diagram showing an example of functional blocks realized in the image orientation device 2. When the CPU 20 of the image orientation device 2 executes the orientation program, the image orientation device 2 realizes a feature point identification unit 30, a corresponding point identification unit 31, an orientation calculation unit 34, a three-dimensional image data creation unit 32, and a display control unit 33.

The feature point identification unit 30 identifies multiple feature points for the first image F1 and each of the multiple second images F2i. A feature point is a point in an image that is visually recognized as a clear feature. A feature point is, for example, a location in an image that corresponds to a corner of a terrain, the end of a white line on a road, a corner of a building, etc.

"Identifying feature points" means identifying the position coordinates of feature points in an image in a photographic coordinate system. Methods for identifying feature points include, for example, a Förstner filter, a SUSAN filter, and a FAST filter. After identifying multiple feature points, the feature point identification unit 30 identifies the position coordinates of each of the multiple feature points in the photographic coordinate system.

Figure 6A is a diagram showing multiple feature points identified in a first image F1. Figure 6B is a diagram showing multiple feature points identified in a second image F21 among multiple second images F2i. In Figures 6A and 6B, the identified feature points are indicated by black circles.

The more feature points identified from the first image F1 and the second image F21, the more preferable. As will be described in detail later, the more feature points identified from the first image F1 and the second image F21, the easier it is to identify many corresponding points from the first image F1 and the second image F21. It is preferable to identify a sufficient number of feature points from the first image F1 and the second image F21 so that at least six corresponding points are identified. A similar procedure is used to identify multiple feature points for other second images F2i (i ≠ 1).

The three-dimensional image data creation unit 32 creates three-dimensional image data of an area including a subject based on the multiple second images F2i. Specifically, the three-dimensional image data creation unit 32 creates three-dimensional image data using the principle of photogrammetry. FIG. 8 is an example of a three-dimensional image obtained from the eight second images F2i shown in FIG. 4B. Note that the three-dimensional image data creation unit 32 may have any configuration.

The display control unit 33 displays the three-dimensional image 32a based on the three-dimensional image data created by the three-dimensional image data creation unit 32 on a display device, which is one of the output devices 24. The user can view the three-dimensional image 32a of the area including the subject from any direction using a keyboard, mouse, etc.

By visually checking the three-dimensional image 32a, the user checks whether the multiple second images F2i provide sufficient data to obtain sufficient accuracy in image orientation by the orientation calculation unit 34, which will be described later. If the three-dimensional image 32a is visually recognized as not reproducing the real subject with sufficient accuracy, it is predicted that it will be difficult to obtain sufficient accuracy in image orientation.

The user also checks whether image orientation of the first image F1 is possible by visually checking the three-dimensional image 32a. If the range of the subject captured in the first image F1 is not included in the three-dimensional image 32a, image orientation of the first image F1 is impossible.

The three-dimensional image 32a may also be associated with absolute coordinates to identify the position coordinates of a point on the subject in the absolute coordinate system. This makes it possible to identify the position coordinates of a corresponding point in the camera coordinate system in the absolute coordinate system. This improves the accuracy of the bundle method with self-calibration (described below), and improves the accuracy of image orientation.

The corresponding point identification unit 31 identifies corresponding points corresponding to each of a plurality of points on the subject (a plurality of points in the absolute coordinate system) from the first image F1 and at least one of the plurality of second images F2i. Here, a "corresponding point" is a point in the photograph coordinate system that corresponds to point P when a certain point on the subject is defined as point P. Corresponding points are also called "tie points."

"Identifying corresponding points" means identifying the position coordinates of corresponding points in the photograph coordinate system.

Specifically, a method in which the corresponding point identification unit 31 identifies corresponding points corresponding to each of a plurality of points in the subject from the first image F1 and the second image F21 will be described. For example, the corresponding point identification unit 31 calculates the similarity for all pairs of a plurality of feature points identified from the first image F1 and a plurality of feature points identified from the second image F21 using a pattern matching method. As the pattern matching method, for example, a normalized cross-correlation method or a least squares correlation method can be used.

Then, the corresponding point identification unit 31 identifies, for example, a pair of feature points whose similarity is equal to or greater than a predetermined threshold, as corresponding points in the first image F1 and the second image F21 that correspond to a certain point on the subject.

Figure 7A is a diagram showing multiple corresponding points identified from a first image F1. Figure 7B is a diagram showing multiple corresponding points identified from a second image F21. In Figures 7A and 7B, the identified corresponding points are indicated by black circles.

In these figures, point q _j (1) in Fig. 7A and point q _j (21) in Fig. 7B both refer to a corresponding point corresponding to point P _j (not shown) in the subject. A plurality of corresponding points are identified in the first image F1 and another second image F2i (i ≠ 1) in a similar procedure.

The orientation calculation unit 34 calculates the orientation elements when the first camera C1 captures the first image F1 based on the position coordinates of each of the multiple corresponding points in the photographic coordinate system. Specifically, the orientation calculation unit 34 uses a bundle method with self-calibration to calculate the orientation elements when the first camera C1 captures the first image F1.

The bundle method is a method for determining exterior orientation parameters by formulating collinearity conditions and solving them using the least squares method. The bundle method with self-calibration is a method for simultaneously calculating interior orientation parameters in addition to the exterior orientation parameters of the camera, and is a method for solving unknown quantities by adding an error model term to the collinearity condition equation used in the bundle method (Equation 3-1 and Equation 3-2).

The following collinearity condition is obtained from one corresponding point q _j (1) among the multiple corresponding points identified from the first image F1. Here, the coordinates of point q _j (1) in the camera coordinate system are (x(1), y(1), z(1)). In addition, the position coordinates in the absolute coordinate system corresponding to corresponding point q _j (1) are (X, Y, Z).

Furthermore, the following formulated collinearity condition is obtained from one corresponding point q _j (21) among the multiple corresponding points identified from the second image F21. Here, the coordinates of point q _j (21) in the camera coordinate system are (x(21), y(21), z(21)). Also, the position coordinates in the absolute coordinate system corresponding to corresponding point q _j (21) are the same as the position coordinates in the absolute coordinate system corresponding to corresponding point q _j (1), and are (X, Y, Z).

In Equation 4-1 and Equation 4-2, variables marked with "(1)" refer to variables related to the first image F1. Similarly, in Equation 5-1 and Equation 5-2, variables marked with "(21)" refer to variables related to the second image F21.

In the same manner, collinearity conditions are obtained twice the number of pairs of corresponding points identified from the first image F1 and the second image F2i (i ≠ 1).

The orientation calculation unit 34 uses the bundle method with self-calibration to calculate the orientation elements when the first image F1 is captured by the first camera C1, using the multiple collinearity conditions formulated by the above-mentioned method.

<Image orientation processing>
An example of the process executed by each functional block will be described below with reference to Fig. 9. Fig. 9 is a flow chart showing an example of the process executed by the image localization device 2.

The image orientation process of this embodiment includes step S1 of acquiring a first image F1, step S2 of acquiring multiple second images F2i, step S2 of identifying feature points, step S3 of identifying corresponding points, and step S4 of calculating orientation elements.

In step S1, the feature point identification unit 30 reads image data for representing the first image F1 that is recorded in the auxiliary storage device 22, and acquires the first image F1.

In step S2, the feature point identification unit 30 reads image data for representing each of the multiple second images F2i recorded in the auxiliary storage device 22, and acquires the multiple second images F2i. Note that the order of steps S1 and S2 is arbitrary.

In step S3, the feature point identification unit 30 identifies multiple feature points from the first image F1 and each of the multiple second images F2i.

In step S4, the corresponding point identification unit 31 identifies multiple corresponding points between the first image F1 and multiple second images F2i.

In step S5, the orientation calculation unit 34 calculates the orientation elements when the first image F1 was captured by the first camera C1, based on the multiple corresponding points obtained in step S2.

==Variation 1==
A modified example of the process executed by each functional block will be described with reference to Fig. 10. Fig. 10 is a flow chart showing a modified example of the process executed by the image localization device 2. The process of this modified example differs from the process of the first embodiment (Fig. 9) in that it further includes step S4' for creating three-dimensional image data. Steps S1, S2, S3, S4, and S5 of the first embodiment are respectively similar to steps S1', S2', S3', S5', and S6' of this modified example.

In this modified example, step S4' of creating the three-dimensional image data is performed before step S5' of identifying corresponding points. In step S4', the three-dimensional image data creation unit 32 creates three-dimensional image data of an area including the subject using a plurality of second images F2i. Note that step S4' may be performed before step S6', and is not limited to the timing of this example.

==Variation 2==
In the processing of the image orientation system 1, it is preferable that the first image F1 and the multiple second images F2i are similar in image quality such as resolution, brightness, contrast, and color, and in the position of shadows, etc. In this way, the accuracy of the pattern matching performed by the corresponding point identification unit 31 can be maintained at a certain level or higher, and the accuracy of the obtained orientation elements can be maintained at a certain level or higher.

Therefore, image processing may be performed on the first image F1 and the multiple second images F2i to adjust the resolution, brightness, contrast, color, etc. to be similar. Such image processing may be performed at any time before step S5 (FIG. 9) of identifying corresponding points.

Second Embodiment
The image orientation system of this embodiment is different from the image orientation system 1 of the first embodiment in that the camera for obtaining the multiple second images F2i is different. The other configurations, processing methods, etc. are the same as those of the first embodiment.

Figure 11 is a diagram showing an example of the arrangement of the first camera C1 and the multiple third cameras C3. In this embodiment, the multiple second images F2i are images captured and acquired by the multiple third cameras C3. Each of the multiple third cameras C3 is fixed in position with respect to the subject.

Each of the multiple third cameras C3 obtains a second image F2i at a predetermined time interval. Each of the multiple third cameras C3 may be a camera for obtaining still images, similar to the second camera C2 in the first embodiment, or a camera for obtaining moving images.

Furthermore, the attitude, shooting angle of view, etc. of each of the multiple third cameras C3 may be remotely controllable, or these may be fixed.

As mentioned above, in the processing of the image orientation system, it is preferable that the first image F1 and the multiple second images F2i are similar in image quality such as resolution, brightness, contrast, and color, and in the position of shadows, etc.

Therefore, it is preferable to select the multiple second images F2i that were obtained as close as possible to the time that the first image F1 was obtained.

In addition, according to the multiple third cameras C3 of this embodiment, the subject can be photographed at any timing to obtain multiple second images F2i. Therefore, the subject may be photographed in advance to obtain multiple second images F2i and create three-dimensional image data. Then, image data for expressing each of the multiple second images F2i and the three-dimensional image data may be stored in the auxiliary storage device 22.

Then, the orientation parameters may be calculated for the first image F1 obtained by the first camera C1 photographing the subject. In this case, in the flowchart of FIG. 10, step S4' of creating the three-dimensional image data may be replaced by a step of reading out the three-dimensional image data recorded in the auxiliary storage device 22.

The above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. Furthermore, the present invention may be modified or improved without departing from the spirit of the present invention, and it goes without saying that the present invention includes equivalents.

As described above, the image orientation methods of the first and second embodiments include the steps of acquiring a first image F1 obtained by photographing the subject with a first camera C1 installed at a fixed position relative to the subject, acquiring a plurality of second images F2i, each of which is obtained by photographing at least a portion of the subject, identifying a plurality of corresponding points between the first image F1 and the plurality of second images F2i, and calculating orientation elements of the first camera C1 at the time of capturing the first image F1 based on the position coordinates in the photographic coordinate system of each of the plurality of corresponding points.

According to this image orientation method, when performing image orientation of the first image F1, it is not necessary to install a marker within the area to be located. Therefore, image orientation can be performed even for images in areas where the installation of markers is prohibited or restricted.

The orientation method may further include a step of creating three-dimensional image data of an area including the subject based on the second images F2i before the step of identifying the corresponding points. This allows the user to visually check the three-dimensional image 32a and confirm whether the second images F2i are sufficient data to obtain sufficient accuracy in image orientation. Also, the user can visually check the three-dimensional image 32a and confirm whether image orientation of the first image F1 is possible. Also, by associating the three-dimensional image 32a with absolute coordinates, the position coordinates of one point of the subject in the absolute coordinate system can be specified. Then, the position coordinates of the corresponding points in the camera coordinate system in the absolute coordinate system can be specified. This improves the accuracy of the bundle method with self-calibration, and improves the accuracy of image orientation.

In addition, in the location method of the first embodiment, the multiple second images F2i are images obtained by photographing the subject with a second camera C2 installed in a position that can be moved relative to the subject. This makes it possible to photograph the subject multiple times from different positions and angles without requiring multiple cameras to obtain the multiple second images F2i.

In addition, in the orientation method of the second embodiment, the multiple second images F2i are images obtained by photographing the subject with multiple third cameras C3 installed at fixed positions. This makes the image quality of the first image F1 and the multiple second images F2i closer. This makes it possible to maintain the accuracy of the pattern matching performed by the corresponding point identification unit 31 at a certain level or higher, and therefore the accuracy of the obtained orientation elements at a certain level or higher.

The image orientation device 2 of the first and second embodiments includes a corresponding point identification unit that identifies corresponding points corresponding to each of a plurality of points on a subject from a first image F1 captured by a first camera C1 whose position is fixed relative to the subject and at least one of a plurality of second images F2i captured by capturing at least a portion of the subject, and an orientation calculation unit 34 that calculates orientation elements of the first camera at the time of capturing the first image F1 based on the position coordinates of the corresponding points corresponding to each of the plurality of points.

With this type of image orientation device 2, when performing image orientation of the first image F1, image orientation can be performed without the need to install markers within the area to be located. Therefore, image orientation can be performed even for images in areas where the installation of markers is prohibited or restricted.

The image orientation system 1 of the first and second embodiments includes a first camera C1 whose position is fixed relative to the subject, a second camera C2, and the image orientation device 2.

With this image orientation system 1, when performing image orientation of the first image F1, image orientation can be performed without the need to install markers within the area to be located. Therefore, image orientation can be performed even for images in areas where the installation of markers is prohibited or restricted.

The image orientation program of the first and second embodiments causes a computer to acquire a first image F1 obtained by photographing the subject with a first camera C1 installed at a fixed position relative to the subject, acquire multiple second images F2i, each of which is obtained by photographing at least a portion of the subject, identify multiple corresponding points between the first image F1 and the multiple second images F2i, obtain position coordinates of each of the multiple corresponding points in a coordinate system fixed to the subject, and calculate orientation elements of the first camera C1 at the time of capturing the first image F1 based on the multiple corresponding points.

According to such an image orientation program, when performing image orientation of the first image F1, it is possible to have the computer perform image orientation without the need to install markers within the area to be located. Therefore, image orientation can be performed even for images in areas where the installation of markers is prohibited or restricted.

1: Image orientation system C1: First camera C2: Second camera C3: Third camera 2: Image orientation device 20: CPU
21: Main memory device 22: Auxiliary memory device 23: Input device 24: Output device 25: Communication device 30: Feature point identification unit 31: Corresponding point identification unit 32: 3D image data creation unit 33: Display control unit 34: Orientation calculation unit

Claims (6)

capturing an image of the object by a first camera, the first camera being fixed in position relative to the object when capturing the image of the object;
capturing a plurality of second images of at least a portion of the subject from a plurality of locations;
creating three-dimensional image data in an absolute coordinate system of an area including the subject based on the plurality of second images;
identifying corresponding points from the first image and at least one of the second images, the corresponding points corresponding to each of the plurality of points in the object;
calculating an orientation element in an absolute coordinate system of the first camera at the time of capturing the first image based on a position coordinate in an absolute coordinate system of the first camera, a position coordinate in a photograph coordinate system of a corresponding point corresponding to each of the plurality of points , and a position coordinate in an absolute coordinate system specified based on the three-dimensional image data of each of the plurality of points;
The image orientation method includes: The image orientation method according to claim 1 , wherein the plurality of second images are images acquired by photographing the subject using a second camera whose position is movable relative to the subject while the second camera is moving. The image orientation method according to claim 1, wherein the plurality of second images are images obtained by photographing the subject using a plurality of third cameras, each of which is fixed in position relative to the subject when photographing the subject. a corresponding point specifying unit that specifies corresponding points corresponding to each of a plurality of points on a subject from a first image obtained by photographing the subject with a first camera whose position is fixed with respect to the subject when photographing the subject, and from at least one of a plurality of second images obtained by photographing at least a portion of the subject from a plurality of locations;
a three-dimensional image data creation unit that creates three-dimensional image data in an absolute coordinate system of an area including the subject based on the plurality of second images;
an orientation calculation unit that calculates orientation elements in an absolute coordinate system of the first camera at the time of capturing the first image, based on position coordinates in an absolute coordinate system of the first camera, position coordinates in a photographic coordinate system of corresponding points corresponding to each of the plurality of points , and position coordinates in an absolute coordinate system specified based on the three-dimensional image data of each of the plurality of points;
An image orientation device comprising: a first camera whose position is fixed with respect to the subject when at least photographing the subject;
A second camera;
The image orientation device according to claim 4 ;
Equipped with
the second camera captures the second plurality of images;
Image orientation system. On the computer,
acquiring a first image by photographing the subject using a first camera that is fixed in position relative to the subject when photographing the subject;
acquiring a plurality of second images obtained by photographing at least a portion of the subject from a plurality of positions;
creating three-dimensional image data in an absolute coordinate system of an area including the subject based on the plurality of second images;
Identifying a plurality of corresponding points from the first image and at least one of the plurality of second images, the corresponding points corresponding to each of the plurality of points in the subject;
Calculating orientation elements in an absolute coordinate system of the first camera at the time of capturing the first image, based on position coordinates in an absolute coordinate system of the first camera, position coordinates in a photographic coordinate system of the plurality of corresponding points, and position coordinates in an absolute coordinate system specified based on the three-dimensional image data of each of the plurality of points.
Image orientation program.

Images