JP2014089078A - Orientation method, orientation program, and orientation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an orientation method, an orientation program, and an orientation device capable of highly accurately obtaining an external orientation element for a group of images in order to realize a highly accurate photogrammetry while moving without using an expensive positioning instrument for survey and an inertia measuring device.SOLUTION: The orientation method is a method for obtaining an orientation element based on a captured image while moving, and includes; a process of imaging; a process of extracting an image; a process of extracting a pair of images; a process of collating images; and a process of orientation. In the process of imaging, two or more imaged images are obtained, and an imaging position and an imaging direction are obtained. In the process of an image conversion, a reference surface image converted into the image of a prescribed reference surface is obtained based on the imaging position and the imaging direction. In the process of extracting a pair of the images, a pair of the images is obtained by extracting two reference surface images including the same portion. In the process of collating the images, the collation of the images is performed using the pair of the images. In the process of orientation, the orientation element is obtained based on the collation result.

Description

  The present invention relates to a technique for measuring a roadside object based on an image of a roadside situation imaged while moving by a moving body, and more specifically, based on a rough orientation element, a deviation of an acquired image The present invention relates to a technique for performing correction and performing matching (matching) of a pair image using the displacement corrected image and obtaining an accurate orientation element from the matching result.

  Conventionally, demand for so-called roadside terrain information (spatial information) such as roads and roadside terrain is high, and measurement techniques have also been advanced to meet the demand.

  Along the road, various facilities such as houses, apartment houses, commercial buildings, retaining walls, signs, signals, outdoor advertisements, etc. are installed. Such structures and facilities are easy to check visually, and it is extremely useful to maintain roadside information to understand landmarks to the destination and the destination itself. . It is also beneficial for those who manage structures and facilities (for example, outdoor advertisements) installed on the roadside because the management work becomes easy.

  On the other hand, land along the road is relatively easy to change due to its high utility value. Especially in urban areas, the replacement of commercial buildings is intense, and outdoor advertisements are also frequently changed. In addition, unexpected topographic changes may occur due to natural disasters. For this reason, it is not necessary to prepare the roadside spatial information once, but it is necessary to redevelop it whenever the topography changes or periodically.

  Traditionally, field surveys using total stations have been the mainstream technology for obtaining spatial information on roadside conditions. According to this method, it takes a lot of labor and time for work, and it costs a lot of money, so it is practically difficult to measure the roadside situation frequently, and spatial information reflecting the latest roadside situation is obtained. It was difficult to maintain.

  In recent years, a technique that is attracting attention as a method that can be carried out more easily than on-site surveying using a total station is a measurement technology that acquires spatial information while moving on a vehicle equipped with a measuring device. This measurement technology is called a mobile mapping system (Mobile Mapping System: MMS), and includes a positioning device (for example, GPS: Global Positioning System), an inertial measurement device (for example, IMU: Internal Measurement Unit), and a distance meter (DMI: A distance measuring instrument), a laser distance meter, an imaging means (for example, a digital camera), and the like are appropriately combined and mounted on a vehicle, and roadside spatial information is acquired by these measuring devices while moving.

  An MMS equipped with a laser rangefinder is to receive and measure the reflected signal of a laser pulse irradiated to a roadside object. Usually, the GPS pulse and IMU are mounted on the vehicle, so the laser pulse irradiation position (X, y, z) and irradiation posture (ω, φ, κ) are obtained, and as a result, the three-dimensional coordinates of the measurement point can be acquired from the time difference between the irradiation time and the reception time. Since MMS that performs laser measurement can acquire a large number of measurement points in one run, it has the effect of grasping the fine topography, but IMU and laser rangefinders are very expensive and can be considered costly. In other words, there was a problem that it was not easy to implement.

  Therefore, Patent Document 1 proposes a technique for measuring a three-dimensional shape of a road by taking a video camera while moving with an observation vehicle without providing a laser distance meter.

JP-A-2002-081941

  Patent Document 1 is a technique for specifying the position coordinates of an object using two photographs (so-called stereo pairs) obtained by photographing the same object from different directions, and so-called photogrammetry that has been performed conventionally. It is.

  When performing photogrammetry with an MMS equipped with imaging means, in addition to obtaining the internal orientation elements of the imaging means such as displacement of the principal point, radioactive distortion, and asymmetric distortion, external orientation elements such as the imaging position and imaging direction at the time of imaging It is necessary to ask. A means for obtaining the external orientation element is a measuring instrument such as GPS, IMU, or DMI. However, high-precision GPS or IMU for surveying is extremely expensive, and the vehicle must be modified to attach the DMI. As a result, compared with MMS equipped with a laser distance meter, the cost can be reduced by not using the laser distance meter, but a significant reduction in cost is not expected.

  By the way, in recent years, GPS and gyros for non-surveying are widely distributed in the market, and are installed as usual in portable terminals such as mobile phones. These non-surveying GPSs and gyros are much cheaper than those for surveying, and a technique for realizing high-precision photogrammetry using them is extremely useful.

  In photogrammetry using MMS, it is conceivable to use a captured image in order to obtain an external orientation element without using a GPS or IMU for surveying. However, in this case, the problems of imaging interval and image distortion can be pointed out. For example, there is a method of obtaining external orientation elements by tracking feature points and matching images for images with a small imaging interval such as moving images, but there is a possibility that errors may accumulate for the number of images. As a result, the expected accuracy may not be obtained. On the other hand, if an external orientation element is obtained from an image group with a large imaging interval, the texture distortion on the image will increase, resulting in unstable feature point extraction and image matching, and may not achieve the expected accuracy. There is.

  An object of the present invention is to provide a technique for solving the above-described problems, that is, to realize high-precision photogrammetry while moving without using an expensive surveying position meter or inertial measurement device. Accordingly, it is an object of the present invention to provide an orientation method, an orientation program, and an orientation device that can obtain an external orientation element with high accuracy for an image group having a large imaging interval.

  The present invention focuses on the point that image collation (matching) is performed with a displacement corrected image obtained by performing displacement correction based on a provisional orientation element, and an accurate orientation element is obtained from the result. It was an invention made based on an idea that did not exist.

  The orientation method of the present invention is an orientation method for obtaining orientation elements based on an image captured during movement, and includes an imaging step, an image extraction step, a pair image extraction step, an image collation step, and an orientation step. . Among these, in an imaging process, while imaging, it acquires two or more captured images, and acquires an imaging position and an imaging direction at the time of imaging. In the image conversion step, each captured image is converted so as to become an image of a predetermined reference plane based on the imaging position and the imaging direction, and a reference plane image is obtained. In the pair image extraction step, two reference plane images including the same location are extracted and used as a pair of pair images. In the image collation process, partial images are collated using the pair images. In the orientation process, orientation elements at the time of imaging in the imaging process are obtained based on the matching result in the image matching process.

  The orientation method of the present invention may be a method including an image position specifying step. In this case, in the image position specifying step, an image position is specified for each reference plane image based on the imaging position and imaging direction at the time of imaging, and in the pair image extraction step, two reference plane images are determined based on the image position. To extract.

  The orientation method of the present invention may be a method of acquiring a front captured image in front of the moving body and a rear captured image in the rear of the moving body in the imaging step. In this case, in the pair image extraction step, a reference plane image is extracted one by one from the front captured image and the rear captured image.

  The orientation program of the present invention is an orientation program that causes a computer to execute a process for obtaining orientation elements based on an image captured during movement, and performs image conversion processing, pair image extraction processing, image collation processing, and orientation processing on the computer. It is a program to be executed. Among these, the image conversion process converts each captured image to be an image of a predetermined reference plane based on two or more captured images captured while moving and the imaging position and imaging direction at the time of imaging. This is a process for obtaining a reference plane image. The pair image extraction process is a process of extracting two reference plane images including the same portion and storing them as a pair of pair images. The image matching process is a process for matching partial images using pair images. The orientation process is an arithmetic process for obtaining orientation elements at the time of imaging based on the matching result obtained by the image matching process.

  The orientation program of the present invention may be a program that causes a computer to execute image position specifying processing. In this case, the image position specifying process is a process of specifying the image position with respect to each reference plane image based on the image pickup position and the image pickup direction at the time of image pickup, and the pair image extraction process is based on two image positions. This is processing for extracting a reference plane image.

  In the orientation program of the present invention, the pair image extraction process may be a process of extracting a reference plane image one by one from a front captured image obtained by imaging the moving front and a rear captured image obtained by imaging the rear of the movement. .

  The orientation device of the present invention is an orientation device that obtains orientation elements based on an image captured during movement, and includes a moving body, an imaging means, a position measuring means, an image converting means, a pair image extracting means, an image collating means, an orientation. A device provided with means. Among these, the imaging means is mounted on the moving body, captures the front of the moving body to acquire a captured image, and captures the rear of the moving body to acquire the rear captured image. The position measuring unit is mounted on the moving body and acquires the position of the imaging unit at the time of imaging. The image conversion means converts the front captured image and the rear captured image into images of a predetermined reference plane based on the imaging position and imaging direction at the time of imaging, and obtains a reference plane image. The pair image extraction unit extracts one front captured image and one rear captured image including the same location one by one, and sets these as a pair of pair images. The image collating means collates partial images using pair images. The orientation means obtains orientation elements at the time of imaging based on the result of matching by the image matching means.

The orientation method, orientation program, and orientation apparatus of the present invention have the following effects.
(1) It is relatively inexpensive because it does not require an expensive surveying positioning meter (such as GPS), inertial measurement device (such as IMU), or a distance meter (such as DMI) that requires modification of a moving object (such as a vehicle). It is possible to obtain spatial information on roadside conditions.
(2) Since the pair images used for matching (correction) are corrected for displacement, the texture distortion on the images is suppressed, so that high-precision image matching is possible, and thus high-precision spatial information is acquired. can do.
(3) If image collation is performed using the “front-captured image” in front of the movement and the “rear-captured image” in the rear of the movement, it is possible to perform image collation with higher accuracy and thus obtain high-accuracy spatial information. .

The flowchart which shows the main flows of this invention. Explanatory drawing which shows an imaging process. Explanatory drawing which shows the imaging process in the case of acquiring the captured image of the front-back 2 directions. The model figure explaining the structure which converts a captured image into a reference plane image. Explanatory drawing which shows the image of a picked-up image, and the image of a reference plane image at the time of imaging the road surface containing a white line. Explanatory drawing which shows the resolution when collating a front reference plane image and a back reference plane image.

  An example of embodiments of the orientation method, orientation program, and orientation device of the present invention will be described with reference to the drawings.

1. General overview When performing photogrammetry, it is necessary to determine orientation elements relating to imaging means and imaging conditions such as a camera and video for acquiring images. As described above, this orientation element includes an internal orientation element (shift of principal point position, radioactive distortion, etc.) provided in the imaging means and an external orientation element (imaging position and imaging direction) that is a condition at the time of imaging. The invention of the present application obtains an orientation element from an acquired image, and performs arithmetic processing based on three-dimensional spatial information. Therefore, first, three-dimensional spatial information will be described.

  The three-dimensional spatial information is information composed of points, lines, surfaces, or combinations thereof having plane coordinate values and height information. Further, the plane coordinate value is represented by latitude and longitude or X and Y coordinates, and the height means a vertical distance from a predetermined reference horizontal plane such as an altitude. Further, the three-dimensional coordinates indicate coordinates expressed by a combination of plane coordinates and height.

  Next, a series of flows of the present invention will be briefly described. FIG. 1 is a flowchart for explaining the present invention. In this figure, actions (processes, processes, means) to be performed are shown in the center column, input information necessary for the action is shown in the left column, and output information generated from the action is shown in the right column. In addition, although FIG. 1 illustrates the orientation method of the present invention and shows each act as a process, in the case of the orientation program of the present invention, each act is processed, and in the case of the orientation device of the present invention, each act is indicated. This figure can be read as a means.

  This will be described with reference to FIG. In the imaging step (Step 10), a plurality of images are acquired, and temporary orientation elements are set (Step 11). At this time, the internal orientation element is obtained from the specifications of the imaging means used, and the external orientation element is obtained by a simple positioning meter (GPS or the like) and an inertial measurement device (IMU, electronic compass, vertical sensor or the like).

  When a plurality of captured images and temporary orientation elements are obtained, the captured images are converted using these as input information (Step 20). Here, the conversion means that the captured image is an image of a predetermined reference plane such as a horizontal plane, and the converted image obtained thereby is referred to as a “reference plane image” for convenience. Since the acquired plurality of captured images are naturally captured from various angles, even the same object is acquired as a deformed image. As a result, a phenomenon in which matching (matching) is not performed well in image matching performed in a later process occurs. In order to prevent this mismatch, a plurality of captured images are converted to a predetermined reference plane (for example, a horizontal plane), that is, to a common plane image.

  When the reference plane images are obtained for all the captured images to be processed, the image position is specified for each reference plane image (Step 30). As a result, each reference plane image is associated with an image position as information. The image position indicates where the reference plane image is actually captured, and is represented by, for example, the plane coordinates of each point constituting the outline of the reference plane image.

  Next, a pair image is extracted (Step 40), and the images are collated (Step 50). The pair image is composed of two reference plane images that partially overlap each other, that is, a pair of pair images is formed by the two reference plane images. When a pair image is extracted, it can be determined by a person, but a suitable pair image may be extracted using the previously obtained image position. Since each reference plane image constituting the pair image has an overlapping portion and the same object is imaged, partial images are collated using the object. As an object in this case, it is preferable to select a characteristic object (hereinafter referred to as “characteristic object”) that can be easily distinguished from others.

  When image collation can be performed for a plurality of features, so-called “orientation” is performed in which orientation elements are obtained by bundle adjustment or the like (Step 60). In this case, both the internal orientation element and the external orientation element can be calculated, but when the internal orientation element obtained from the specifications of the imaging means is reliable, only the external orientation element is used using this internal orientation element. It can also be calculated. What is calculated here is a highly accurate orientation element obtained through various adjustments. If a highly accurate orientation element is obtained, three-dimensional coordinates of various features shown in the acquired captured image can be obtained (Step 70).

  Hereinafter, the orientation method, orientation program, and orientation device of the present invention will be described in detail for each constituent element.

2. Imaging The imaging process which comprises the orientation method of this invention is demonstrated. FIG. 2 is an explanatory diagram showing an imaging process. As shown in this figure, a plurality of images are picked up by the image pickup means 20 mounted on the moving body 10 while moving by the moving body 10 such as a car, a train, a bicycle, and an aircraft. A captured image is acquired every time the moving body 10 advances by a predetermined distance or a predetermined time. As an imaging interval, an image is taken every time the moving body advances 10 cm, or every time the moving body advances 3 m. An image can be taken every time one second elapses, or can be appropriately designed according to the situation. However, at least the captured images adjacent in the traveling direction need to have an imaging interval such that a part thereof overlaps. As described above, since the accumulation of errors increases when the imaging interval is too small, it is desirable to set the imaging interval to 2 to 5 m.

  FIG. 3 is an explanatory diagram illustrating an imaging process when acquiring captured images in two front and rear directions. As shown in this figure, with the two imaging means 20 mounted on the moving body 10, a front captured image in the moving direction (hereinafter referred to as “front captured image”) and a rear captured image in the moving direction (hereinafter “ It is also possible to acquire two captured images “backward captured images”). In this case, if the imaging interval is such that the front captured image and the rear captured image partially overlap, it is not always necessary to set the interval so that adjacent front captured images overlap. In FIG. 2, two image pickup means 20 (that is, the front image pickup means 21 and the rear image pickup means 22) are installed in order to acquire the front image pickup image and the rear image pickup image. If an azimuth camera or the like is used, one image pickup means 20 can be installed.

  When acquiring a captured image, an imaging position and an imaging direction at the time of shooting (shooting timing) are measured. The imaging position is a three-dimensional coordinate (for example, X, Y, Z) of the principal point position of the imaging unit 20, and the imaging direction is a direction captured by the imaging unit 20, for example, a rotation angle (ω from three axes). , Φ, κ, etc.). The imaging position can be measured by a positioning meter (GPS or the like), and the imaging direction can be measured by an inertial measurement device (IMU or the like). However, the imaging position and imaging direction obtained and measured here are set as temporary external orientation elements, and may be approximate values without requiring so high accuracy. Therefore, the positioning device used here can be a simple GPS (not for surveying). Also, the inertial measurement device need not be an expensive IMU, and an electronic compass or a vertical sensor can be used as the inertial measurement device, or an inertial measurement device using GPS. The inertial measurement device using GPS is a mechanism for obtaining the posture of the imaging unit 20 based on the three-dimensional coordinates before and after the movement of the moving body 10. For example, the imaging unit 20 uses GPS mounted at three locations on the moving body 10. It is possible to estimate the imaging direction by obtaining the attitude of the imaging means 20 by various methods such as obtaining the attitude of the image.

  The moving body 10, the imaging unit 20, the positioning meter (position measuring unit), and the inertia measuring device described here constitute the orientation device of the present invention.

3. Image Conversion An image conversion process constituting the orientation method of the present invention, an image conversion process constituting the orientation program of the present invention, and an image conversion means constituting the orientation method of the present invention will be described. As described above, since the captured images are acquired in various imaging directions, even if two captured images have the same feature, each feature has a unique deformation, so image matching is successful. Sometimes it doesn't. Therefore, if the captured images to be collated are converted into a common plane and image collation is performed, a result of collation can be obtained with relatively high accuracy.

  FIG. 4 is a model diagram illustrating a mechanism for converting a captured image into a reference plane image. As shown in this figure, the captured image obtained by imaging by the imaging means 20 is actually projected on the imaging plane P. The captured image at this time is captured using the “imaging range” shown in the figure as a real space. As a result of estimating the actual imaging range and assuming that the range is captured by the virtual imaging means 20v from vertically above, the obtained image is the reference plane image. In other words, the reference plane image is obtained by converting the captured image projected on the imaging plane P into an image projected on the reference plane S (in this case, the horizontal plane). Here, it is assumed that the virtual imaging means 20v is imaged from above, and the reference plane S is a horizontal plane. However, the present invention is not limited to this, and a plane having an arbitrary inclination can be selected as the reference plane S.

ここで、左辺にある（ｘ，ｙ）は撮像画像上の任意座標系における平面座標で、右辺にある（Ｘ，Ｙ，Ｚ）は実空間の３次元座標である。また、（Ｘ_０，Ｙ_０，Ｚ_０）は撮像手段２０の主点位置の３次元座標、△ｘと△ｙは内部標定要素の補正量、ｃは撮像手段２０の焦点距離、ａ_１１〜ａ_３３は外部標定要素（撮像手段２０の姿勢）によって定まる定数である。 The reference plane image is created by correcting the displacement of the captured image, and is performed in the manner of creating a so-called ortho image (orthographic projection image). Specifically, deviation correction is performed using the following equation.

Here, (x, y) on the left side is a plane coordinate in an arbitrary coordinate system on the captured image, and (X, Y, Z) on the right side is a three-dimensional coordinate in the real space. (X ₀ , Y ₀ , Z ₀ ) is the three-dimensional coordinate of the principal point position of the imaging means 20, Δx and Δy are the correction amounts of the internal orientation elements, c is the focal length of the imaging means 20, and a ₁₁ to a ₃₃ is a constant determined by the external orientation element (the posture of the imaging means 20).

In order to perform the above conversion, orientation elements are required. Internal orientation elements use values obtained from the specifications of the imaging means 20 and external orientation elements use the provisional imaging position and imaging direction acquired earlier. use. In addition, since the height of the object to be imaged (the road surface in the case of FIGS. 2 and 3) must also be known, a temporary height is set with reference to the height of the moving body 10 and the like. A difference in specific height depending on the assumed height of the imaging target and the height of the imaging means 20 is indicated by a symbol h in FIG. If the orientation element is given and the specific height difference h in FIG. 4 is given (that is, if Z = h in the above equation), the constants a _{11 to} a ₃₃ in the above equations (1) and (2) are changed. As a result, a reference plane image projected on the reference plane S can be obtained.

  FIG. 5 is an explanatory diagram illustrating an image of a captured image and an image of a reference plane image when a road surface including a white line is captured. As shown in this figure, the shape of the white line is deformed in the captured image, and the greater the distance from the main point of the imaging means 20, the greater the distortion. On the other hand, white lines are displayed in the reference plane image with almost no deformation as taken from right above. From this figure, it can be seen that it is difficult to collate with other images if the captured image is used as it is, but if the reference plane image is used, it is easier to perform image collation.

  The image conversion step constituting the orientation method is a step of performing the image conversion described here, and can be executed by a person, or can be processed by an operator operating the orientation program and orientation device of the present invention. . Further, the image conversion process constituting the orientation program is a process for causing the computer to execute the image conversion described here. Further, the image conversion means constituting the orientation device is a means for executing the image conversion described here by a computer or the like.

4). Pair Image Extraction A pair image extraction process constituting the orientation method of the present invention, a pair image extraction process constituting the orientation program of the present invention, and a pair image extraction means constituting the orientation method of the present invention will be described. In order to perform image collation, a pair of reference plane images (pair images) called stereo pairs are extracted.

  In the pair images, a part of the reference plane images overlap each other, that is, each reference plane image is an image including the same portion. However, since a large number of captured images, that is, reference plane imaging is usually obtained by one measurement, it is not easy to extract an appropriate pair image from them. Although it is conceivable that serial numbers are assigned to the reference plane images in the order in which they are captured and the continuous reference plane images are used as pair images, the overlapping range (hereinafter referred to as “wrap length”) is too large, which is inefficient. For example, adjacent reference plane images are not necessarily suitable for pair images. Therefore, an image position is assigned to each reference plane image, and a suitable pair image is extracted based on the image position. Since the reference plane image is obtained through the above-described deviation correction, the plane coordinates such as points constituting the outline of the reference plane image have already been calculated, and these are used as image positions for each reference plane image. Give. As a method for extracting a suitable pair image based on the image position, for example, there is an extraction method based on coordinate calculation such as extracting a combination whose wrap length is close to a reference value (for example, 60% of the image area) as a pair image. Can be mentioned. If a method of extracting pair images by coordinate calculation is used, automatic extraction using a computer or the like is also possible. It should be noted that the provision of the image position to the reference plane image described here is performed in the image position specifying step that constitutes the orientation method, and is executed by the computer based on the image location specifying process that constitutes the orientation program. It is executed by the image position specifying means that constitutes.

  Two pairs of images can be extracted from the reference plane image (hereinafter referred to as “front reference plane image”) based on the above-described front captured image. It is more preferable to include “back reference plane image”). That is, one sheet is extracted from the front reference plane image, one sheet is extracted from the rear reference plane image, and these two reference plane images are used as a pair image.

  FIG. 6 is an explanatory diagram showing the resolution when the front reference plane image and the rear reference plane image are collated. As shown in this figure, the farther away from the main point of the image pickup means 20, the smaller the same feature (for example, white line in this figure) is displayed, and the lower the resolution is. In the case of a front reference plane image, the resolution decreases as it goes forward in the image. Conversely, in the case of a rear reference plane image, the resolution decreases as it moves backward in the image. When a pair image is extracted from only the front reference plane image, the front range of one image overlaps the rear range of the other image, that is, one image is subjected to image collation at a low resolution portion. As a result, it is conceivable that the accuracy of image matching is lowered. As shown in FIG. 6, when a pair image is extracted from the front reference plane image and the rear reference plane image, the rear range of the front reference plane image and the front range of the rear reference plane image overlap, that is, both images have a resolution. The image collation is performed at a high part. As a result, the accuracy of image matching is improved.

  The pair image extraction step constituting the orientation method is a step of extracting the pair image described here, and can be performed by human judgment, or the operator operates the orientation program or orientation device of the present invention. It can also be processed. Moreover, the pair image extraction process which comprises an orientation program is a process which makes a computer perform the image conversion demonstrated here. Further, the image conversion means constituting the orientation device is a means for executing the image conversion described here by a computer or the like.

5. Image Matching An image matching process constituting the orientation method of the present invention, an image matching process constituting the orientation program of the present invention, and an image matching means constituting the orientation method of the present invention will be described. The two reference plane images constituting the extracted pair image (stereo pair) have portions that overlap each other, and the same object is imaged therein. Using the object as a clue, partial image matching is performed. A plurality of portions to be collated are detected for one pair image. As described above, it is preferable to extract a large number of features that can be easily distinguished from others and to collate with these features. In addition, the image collation performed here can utilize the technique currently performed conventionally, and can apply the image recognition technique using a computer, and can detect an object automatically, and can also perform image collation by this .

  The image collation process constituting the orientation method is the image collation process described here, and can be extracted by human judgment, or can be processed by the operator operating the orientation program or orientation device of the present invention. You can also. Moreover, the pair image extraction process which comprises an orientation program is a process which makes a computer perform the image collation demonstrated here. Further, the image conversion means constituting the orientation device is means for executing the image collation described here by a computer or the like.

6). Orientation The orientation process that constitutes the orientation method of the present invention, the orientation process that constitutes the orientation program of the present invention, and the orientation means that constitutes the orientation method of the present invention will be described. Orientation is based on the results of image matching described above, including block adjustment using the bundle method that has been used in the past, mutual orientation, single course adjustment, block adjustment using the polynomial method, and block adjustment using the independent model method. , DLT method or the like. The result obtained by this orientation is the final orientation element. As described above, both the internal orientation element and the external orientation element can be calculated, but when the internal orientation element obtained from the specifications of the imaging means is reliable, the external orientation can be calculated using this internal orientation element. Only elements can be calculated.

  The orientation process that constitutes the orientation method is the step of performing orientation as described here, and can be extracted by human judgment, or can be processed by the operator operating the orientation program or orientation device of the present invention. . Further, the orientation processing that constitutes the orientation program is processing for causing the computer to execute the orientation described here. Further, the image conversion means constituting the orientation device is a means for executing the orientation described here by a computer or the like.

  The orientation method, orientation program, and orientation device of the present invention are effective inventions for creating and updating maps because roadside spatial information can be obtained at low cost and with high accuracy. Moreover, since outdoor advertisements and road occupants installed along the road can be easily and accurately grasped, it is an extremely useful invention for roadside facility managers. Furthermore, it is an invention that can be used in various mobile objects such as not only roads but also easily applied to situations along tracks (tracks).

DESCRIPTION OF SYMBOLS 10 Moving body 20 Imaging means 21 Front imaging means 22 Rear imaging means 20v Virtual imaging means h Specific height difference P Imaging surface S Reference surface

Claims (7)

In an orientation method for obtaining orientation elements based on images taken during movement,
An imaging step of acquiring two or more captured images by imaging while moving, and acquiring an imaging position and an imaging direction at the time of imaging;
Based on the imaging position and the imaging direction, each of the captured images is converted to be an image of a predetermined reference plane, and an image conversion step for obtaining a reference plane image;
A pair image extraction step of extracting the two reference plane images including the same location and making these a pair of images;
An image matching step for matching a partial image using the pair image;
And an orientation step for obtaining orientation elements at the time of imaging in the imaging step based on the matching result of the image matching step. An image position specifying step of specifying an image position for each of the reference plane images based on the image pickup position and the image pickup direction at the time of image pickup;
The orientation method according to claim 1, wherein in the pair image extraction step, the two reference plane images are extracted based on the image position. In the imaging step, the imaging means mounted on the moving body obtains a front captured image in front of the movement and a rear captured image in the rear of the movement,
The orientation method according to claim 1 or 2, wherein in the pair image extraction step, the reference plane image is extracted one by one from the front captured image and the rear captured image. In an orientation program that causes a computer to execute a process for obtaining orientation elements based on an image captured during movement,
Based on two or more picked-up images picked up while moving and the pick-up position and pick-up direction at the time of pick-up, each picked-up image is converted to become a predetermined reference face image to obtain a reference face image. Image conversion processing,
A pair image extraction process for extracting two reference plane images including the same portion and storing them as a pair of pair images;
Image collation processing for collating partial images using the pair images;
An orientation program comprising a function for causing the computer to execute orientation processing for obtaining orientation elements at the time of imaging based on a matching result obtained by the image matching processing. A function of causing the computer to execute an image position specifying process for specifying an image position for each of the reference plane images based on the imaging position and the imaging direction at the time of imaging;
The orientation program according to claim 4, wherein in the pair image extraction process, the two reference plane images are extracted based on the image position.   5. The paired image extraction process is characterized in that the reference plane image is extracted one by one from a front captured image obtained by imaging the front of the movement and a rear captured image obtained by imaging the rear of the movement. Item 5. The orientation program according to item 5. In an orientation device that obtains orientation elements based on images captured during movement,
A moving object,
An imaging unit mounted on the mobile body, capturing a front captured image by capturing the front of the mobile body, and capturing a rear captured image by capturing the rear of the mobile body;
A position measuring means mounted on the moving body for acquiring the position of the imaging means at the time of imaging;
Image conversion means for converting the front captured image and the rear captured image into images of a predetermined reference plane based on an imaging position and an imaging direction at the time of imaging to obtain a reference plane image;
A pair image extraction unit that extracts the front captured image and the rear captured image including the same part one by one, and sets these as a pair image;
Image collating means for collating partial images using the pair images;
An orientation device comprising orientation means for obtaining orientation elements at the time of imaging based on the result of matching by the image matching means.

Images