JP5084374B2 - Position measuring apparatus and position measuring method - Google Patents

Position measuring apparatus and position measuring method Download PDF

Info

Publication number
JP5084374B2
JP5084374B2 JP2007179036A JP2007179036A JP5084374B2 JP 5084374 B2 JP5084374 B2 JP 5084374B2 JP 2007179036 A JP2007179036 A JP 2007179036A JP 2007179036 A JP2007179036 A JP 2007179036A JP 5084374 B2 JP5084374 B2 JP 5084374B2
Authority
JP
Japan
Prior art keywords
obtained
point
tracking
feature
corresponding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2007179036A
Other languages
Japanese (ja)
Other versions
JP2009014628A (en
Inventor
哲治 穴井
伸夫 高地
仁志 大谷
Original Assignee
株式会社トプコン
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社トプコン filed Critical 株式会社トプコン
Priority to JP2007179036A priority Critical patent/JP5084374B2/en
Priority claimed from US12/166,660 external-priority patent/US7844077B2/en
Publication of JP2009014628A publication Critical patent/JP2009014628A/en
Application granted granted Critical
Publication of JP5084374B2 publication Critical patent/JP5084374B2/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Description

  The present invention relates to a position measuring device and a position measuring method. More specifically, the present invention relates to a position measurement device and a position measurement method for obtaining a three-dimensional coordinate of an imaging device or an object from a dynamic image of the object obtained by moving the imaging device relative to the object.

  There is a technique in which an imaging device continuously photographs an object while moving relative to the object, and measures the self position and the position of the object using the obtained images. This is to track (track) the corresponding points between multiple shot images taken at a certain distance, create a stereo pair from the two shot images, and create a model image of these multiple models. The bundle is adjusted based on the image to obtain the final three-dimensional measurement result. As such a technique, for example, a technique as disclosed in Patent Document 1 is disclosed.

Japanese Patent Application No. 2005-355470

  In the above conventional technique, feature points are extracted from the obtained captured images, and candidate corresponding points corresponding to the feature points are tracked in real time for a plurality of captured images obtained before and after the captured images, and the corresponding results are used. The point is determined, and the photographing position or the coordinates of the object are calculated using the determined corresponding point. In this process, as will be described later, relative orientation for obtaining a model coordinate system so that two captured images that form a stereo pair satisfy conditions (coplanar conditions, collinear conditions, vertical parallax removal method, etc.) to be satisfied Furthermore, in order to obtain the moving process of the photographing apparatus, an operation such as bundle adjustment for adjusting a plurality of model coordinates is performed, but each processing / calculation process is performed with the maximum likelihood coordinates from a large number of observation data including errors. Accompanied by operations for obtaining values and coordinate conversion parameters. Conventionally, the least square method is often used in these operations.

However, when the self-position of the imaging device and the 3D coordinates of the object to be photographed are finally obtained, if there is a miscorresponding point in the feature point, not only the 3D coordinates can be obtained accurately, but in some cases In some cases, the solution does not converge during the calculation process, and as a result, tracking cannot be performed.
Errors in the tracking unit include, for example, miscorresponding points and corresponding points that are not appropriate for calculation (for example, points that move slightly (leaves of trees, etc.), pseudo corresponding points: points that seem to match) In such a case, the calculation accuracy deteriorates in the relative orientation calculation or the like in the subsequent stage, and in the worst case, the solution does not converge. Further, the same applies when a stereo pair containing a large number of them is selected. If the convergence of the solution becomes worse, the calculation speed decreases. If the model coordinates can be calculated and the error is still included in the subsequent bundle adjustment, the accuracy may be uncertain because the large error cannot be removed. In addition, if there are many errors or some points with large errors, estimation calculation such as the least squares method is performed, so it takes time to converge and cannot be processed in real time. was there.

  Accordingly, an object of the present invention is to provide an apparatus and a method that can perform three-dimensional measurement of a self-position or an object at high speed and with high accuracy.

  In order to solve the above-described problem, the position measuring device according to claim 1 is, for example, a captured image of a video camera or the like in which the relative position with respect to the object to be photographed changes with time as illustrated in FIG. A feature extraction unit (3) for extracting a plurality of feature points of a subject to be photographed in an image from a series of photographed images composed of a plurality of frames obtained from the acquisition means (2), and a plurality of the photographed images, The corresponding point corresponding to the feature point is searched, the feature point tracking unit (4A) for tracking the feature point, the feature point of one image and the other from the tracking process or result of the feature point tracking unit (4A) Based on the positional relationship of the feature points on the image, robust estimation is performed on the residual of the coordinate value obtained using the projection parameter between the one image and the other image, and the estimated value of the projection parameter is calculated. First tracking determination unit (4B) to be obtained and first tracking Using the projection parameter estimated value obtained at the section (4B), the residual of the coordinate value from the predicted value of the corresponding point of each feature point is obtained, threshold processing is performed, and each feature point is an incorrect corresponding point. The estimated values of the projection parameters obtained by the second tracking determination unit (4C) and the first tracking determination unit (4B) for excluding erroneous correspondence points from the feature points are determined. The re-template matching unit (4D) that obtains the coordinate value of the estimated position of the erroneous corresponding point, performs template matching near the position, and obtains the coordinate value of the appropriate corresponding point, and the second tracking determination Based on the coordinate value of each corresponding point other than the erroneous corresponding point obtained by the unit and the coordinate value of the appropriate corresponding point obtained by the template matching unit (4D), or the photographed image The shooting position or shooting posture of the acquisition means (2) And a position measurement section 7 for measuring.

  Here, the robust estimation can be performed relatively easily even when observation data includes a large error. For example, the LMedS method, the median estimation (M estimation) method, the Ranzac (RANSAC) method. Is applicable. The threshold processing is processing for excluding or selecting data depending on the magnitude of the threshold compared with a preset threshold. With this configuration, since a large error is removed before each estimation calculation process, the calculation converges quickly and a highly accurate result can be obtained. The “tracking process” means that the tracking result is received in real time, and the “from (tracking) result” means that the tracking result is accumulated and is not processed later. Saying to process in real time.

  The invention described in claim 2 clearly indicates that the robust estimation of claim 1 is any one of the LMedS method, the median estimation (M estimation) method, and the RANSAC method. With this configuration, a large error can be removed by a simple process with less burden before each estimation calculation process, so that the calculation converges quickly and a highly accurate measurement result can be obtained.

  According to a third aspect of the present invention, in the first or second aspect, for example, as shown in FIG. 1 and FIG. 3B, the first tracking determination unit (4A) selects from a plurality of acquired feature points. A projection parameter between the shooting screens is obtained from the plurality of partial feature points (S232), the coordinates of the corresponding points of the feature points other than the partial feature points are calculated from the projection parameters, and the coordinates are used as a reference. The residual of the coordinate value of the feature point is obtained (S233). With this configuration, the conversion characteristics between the images can be easily estimated, and a large error can be removed by a simple process before each estimation calculation process, so that the calculation converges quickly and a highly accurate result can be obtained. .

  Furthermore, the invention according to claim 4 is the projector according to claim 1, wherein the projection parameter is an affine transformation parameter. If comprised in this way, it can respond easily to both the translation of a coordinate and the coordinate conversion of rotation.

  According to a fifth aspect of the present invention, in the third or fourth aspect, for example, as shown in FIG. 3B, the robust estimation in the first tracking determination unit adopts the LMedS method (S234 to S235), and the part Based on the projection parameters obtained from the corresponding points, the median value of the difference between the coordinate values is obtained for each corresponding point (S234), the same processing is performed for the combination of the other partial corresponding points, and the obtained median The combination of the partial corresponding points that minimizes the value is selected, and the projection parameter is obtained (S236). With this configuration, since a large error can be removed by a simple process before each estimation calculation process, the calculation converges quickly and a highly accurate result can be obtained. Here, the magnitude of the difference between the coordinate values only needs to be positive, and therefore the square of the coordinate values may be used.

  The invention described in claim 6 includes, for example, a GPS position data acquisition unit (12) that obtains shooting position data from GPS data in claims 1 to 5, as shown in FIG. The position measurement unit (7) associates the data with the shooting position data of each frame. With this configuration, the validity of the measurement result can be confirmed and the reliability of the measurement result can be improved.

  The invention according to claim 7 is an invention of a method corresponding to claim 1, and for example, as shown in FIGS. 1 and 3A (mainly, FIG. 3A), the relative position to the object to be photographed is Feature extraction step (3) (S21) for extracting a plurality of feature points of a photographing object in an image from a series of a plurality of frames obtained from a photographed image acquisition means (2) that changes over time. A feature point tracking step (S22) for searching for a corresponding point corresponding to the feature point in the plurality of captured images and tracking the feature point, and a tracking process or result of the feature point tracking step (S22) Based on the positional relationship between the feature point of one image and the feature point on the other image, robust estimation is performed for the residual of the coordinate value obtained using the projection parameters between the one image and the other image. First tracking to obtain an estimate of the projection parameter Using the estimated values of the projection parameters obtained in the disconnecting step (S23) and the first tracking determination step (S23), a residual coordinate value from the predicted value of the corresponding point of each feature point is obtained. A second tracking determination step (S24) in which threshold processing is performed to determine whether each feature point is a miscorresponding point, and the miscorresponding point is excluded from the feature point; and the first tracking determination step Re-template for obtaining the coordinate value of the estimated position for the erroneous corresponding point using the estimated value of the projection parameter obtained in (S23), performing template matching near the position, and obtaining the coordinate value of the appropriate corresponding point Based on the matching step (S25), the coordinate value of each corresponding point other than the erroneous corresponding point obtained by the second tracking determination unit, and the coordinate value of the appropriate corresponding point obtained in the template matching step (S25). The shooting object Coordinates, or, photographing position of the photographed image acquisition means (2), or, and a position measurement step of measuring the photographing position (7).

  With this configuration, since a large error is removed before each estimation calculation process, the calculation converges quickly and a highly accurate result can be obtained.

  Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 9 is a diagram for explaining the concept in the present embodiment. The camera 2 is mounted on the car, the position of the car is changed little by little, and the city area that is the object is photographed. Characteristic landmarks (characteristic points 101a and 101b: for example, specific points of the characteristic building) In this example, the position coordinates of the camera 2, that is, the trajectory of the car is obtained from the tracking result of This makes it possible to continuously display the position of the car in the car navigation, but it is also significant to be used complementarily in a section where GPS radio waves cannot be received. When a camera 2 (hereinafter also referred to as a captured image acquisition unit 2) is mounted on a vehicle as shown in the figure, the imaging screen swings, for example, rotated left / right / up / down or in a certain direction. The captured image often includes moving objects such as other cars and people. These all cause errors.

[Hardware and software module configuration]
FIG. 1 shows a configuration example of a position measuring apparatus 100 in the present embodiment. FIG. 1 schematically shows hardware and software modules mounted on the hardware. In the figure, an operation unit 1 is a part that controls each unit of the position measurement device 100 to function as a position measurement device. Specifically, the operation unit 1 instructs the photographed image acquisition unit 2 to start and stop photographing, and a feature extraction unit. 3 setting change, tracking unit 4 setting change, start / stop instruction, relative orientation unit 5, bundle adjustment unit 6 processing start instruction, orientation execution instruction and the like.
The captured image acquisition unit 2 is a part that sequentially acquires captured images such as moving images, and is typically a video camera. In addition to acquisition of the captured image, output to the feature extraction unit 3, storage of the captured image in the moving image memory 10, and the like are performed. Note that the stored data may be all captured frames or thinned data. Further, the position measurement apparatus 100 may not have the captured image acquisition unit 2 and may acquire a captured image from an external imaging apparatus by a communication means such as a cable or wireless.

The feature extraction unit 3 extracts feature points in the image from the slightly different captured images sequentially acquired. The feature extraction unit 3 extracts the feature points from the captured image input from the captured image acquisition unit 2 and obtains the obtained feature points. Is output to the tracking unit 4 and the relative orientation unit 5.
The tracking unit 4 searches for another frame image from the density information of the feature point input from the feature extraction unit 3 and searches for a point (corresponding point) having the same density information around the point. Keep track of. In addition to this tracking, the tracking result is output to the relative orientation unit 5, the execution start instruction, the determination of the arrangement of candidate corresponding points, the feature extraction unit 3 is instructed to establish a new feature point, and the like.
In the following description, “feature point” refers to a point obtained from the feature point extraction unit, and “corresponding point” refers to the same shooting target between two or more frame images among the feature points. Points corresponding to each other between feature points confirmed to be.

  In FIG. 1, the processing in the captured image acquisition unit 2, the feature extraction unit 3, and the tracking unit 4 is typically a part that performs processing in real time when image data is input. 5 and the bundle adjustment unit 6 are typically non-real-time processes that accumulate and process data over a plurality of images obtained by the feature extraction unit 3 and the tracking unit 4 in a memory. However, if necessary, the tracking process may be a non-real time process, and the relative orientation unit 5 and the bundle adjustment unit 6 may be performed in real time.

Information of feature points serving as a reference for these processes is stored in the feature point information memory 9. In this memory, information such as the coordinate values of the feature points of each frame and the correspondences (corresponding points) with the feature points of other frames can be written / erased at any time during the process. Reference numeral 10 denotes a moving image memory for storing captured images. The position measurement unit 7 outputs final three-dimensional data based on the coordinate values of the processed feature points (or corresponding points), external orientation elements, and the like. This result is output to the display unit 8 and the external output unit 11.
The display unit 8 displays, for example, a three-dimensionally measured image of the target object in a three-dimensional manner.

[Overall flow]
FIG. 2 shows an example of the overall flow of the image processing method according to the first embodiment. still. Details of each partial process will be described later. First, the captured moving image data is sequentially fetched (S10). These are a series of frame images. Next, a tracking process is performed (S20). That is, a feature point is extracted from an image of one frame of the captured moving image, and a pattern such as light and shade including the vicinity of the feature point is stored. Here, using a pattern of feature points in the frame image as a template, a similar pattern portion is obtained in an image of another frame, and corresponding points are obtained. Next, it is evaluated whether or not the plurality of corresponding points obtained in this way are appropriate corresponding points, and the erroneous corresponding points are excluded from the feature point information memory 9 or information to that effect is written (hereinafter referred to as the corresponding corresponding points). Simply say “exclusions”). The process of S20 is typically a real-time process. The input frame is sequentially processed, the corresponding point is saved each time, and the feature point information memory 9 stores the result of the evaluation as to whether or not it is an incorrect corresponding point. Rewrite. Of course, the input frame can be processed by thinning out as appropriate. As described above, the corresponding point (temporary corresponding point) serving as the measurement reference point is determined (S30). Here, “temporary” is used because it may be excluded as a result of re-evaluation of the appropriateness of corresponding points in subsequent processing (mutual orientation, bundle adjustment, etc.).

  Next, in S40, a relative orientation process is performed to align each coordinate axis between corresponding points of the two image data. That is, an appropriate stereo pair for relative orientation is selected, and a three-dimensional image is obtained by adjusting the coordinate axes of the images so as to satisfy, for example, coplanar conditions between them based on the corresponding points determined in S30. Like that. If there is an inappropriate corresponding point that can be found in the relative orientation process, the feature point information memory is rewritten and the data is removed. As a result, a stereo pair suitable for three-dimensional coordinate determination, such as having a sufficient baseline length, is determined (S50).

Next, connection orientation is performed to obtain continuous model images (S55). This is a process for unifying the inclination and scale between the models to make the same coordinate system. Then, bundle adjustment processing is performed to obtain continuous model images in which the external orientation elements of each image are simultaneously determined by the least square method using tie points and pass points included in the images (S60). Here too, the corresponding points that give a large error in the course of this processing are excluded, and then the final bundle adjustment is performed, and the external orientation elements such as the position coordinates and orientation of the captured image acquisition unit 2, and the tertiary Original coordinates are obtained (S70).
Hereinafter, each process will be described in more detail.

[tracking]
FIG. 3A shows the processing contents of the tracking processing unit 4. The processing contents will be described with reference to FIG. As shown in FIG. 1, first, the captured image acquisition unit 2 acquires a captured image. The image may be acquired by capturing the image with its own image capturing device, or the image captured with another image capturing device may be acquired via a communication line, memory, tape, or the like. Next, captured images that change little by little are sequentially supplied to the feature extraction unit 3. In this embodiment, since the camera 2 is mounted on the automobile and the camera is photographed while moving, the photographed image that changes little by little is a photographed image that changes little by little in terms of time, and therefore spatially. (Alternatively, the object is commonly included in most of the images). However, the camera 2 may be not only a car but also a person walking with it or taking a picture on a bicycle.

[Feature point extraction]
Feature extraction (S21 in FIG. 3A) is performed in the feature extraction unit 3. Typically, the initial frame is extracted from the entire screen, and the next frame is extracted from a new screen area that does not overlap the initial frame. Alternatively, for example, when the number of points of the template is reduced by 30 to 50% at a certain timing, the feature may be extracted and a new feature point may be generated, and tracking may be performed while updating the template. That is, it is not always necessary to generate feature points every time. For extracting feature points, for example, MORAVEC operator (HP Moravec. Towers Automatic Visual Observed Aviation, Proc. 5th International Joint Conf. On P. Can be adopted as appropriate.

  Whatever feature extraction operator is used, there is a problem that it is easy to react to slight noise on the image (such as noise on the edge). In order to improve this property, noise removal processing of an image is performed using a noise filter such as an average value filter before using the feature extraction operator.

  Depending on the subject to be photographed, feature points may be concentrated on a certain part of the image (such as a tree or grass). This means that the entire screen is evaluated from some information on the screen, which may adversely affect template matching described later. In order to avoid this, a collocation process is performed. Furthermore, when the maximum number of feature points is designated in advance, the feature points are arranged evenly over the entire image, so that the relative orientation and the like of the subsequent processing can be ensured. As described above, the feature point extraction is executed through various processes, but since it is not directly related to the present invention, further explanation is omitted.

[Tracking corresponding points]
The description will be continued with reference to FIG. 3A. The tracking process (S22) is a process performed by the feature point tracking unit 4 and performs a tracking process between frames corresponding to each feature point selected by the feature extraction process. That is, it is a process for obtaining candidate corresponding points in another image corresponding to the feature points. Specifically, a search (template matching) is performed for a portion of the adjacent captured image where the surrounding image pattern approximates the shading pattern around the feature point, and a candidate corresponding point corresponding to one feature point is obtained. As template matching, for example, the SSDA method (sequential residual method) can be used. The SSDA method (sequential residual method) is a method for determining the similarity of a pattern using a difference in density of corresponding points, and the position where the difference in density in the comparison range (template size) is minimized. As a candidate corresponding point. In addition, other methods such as a normalized correlation method can also be employed as template matching (see, for example, Akiyama Minoru, “Photogrammetry”, published by Sankaidou Co., Ltd., PP234-235). For template matching, it is important to optimally select a template size and a search range, and the search range is optimally set based on the frame rate, moving speed, and the like of the video camera 2.

  Candidate corresponding points for each feature point are obtained by template matching, but many mismatches also occur. For example, errors in SSDA template matching itself, misappropriate points due to mismatching, such as when feature points are given to moving objects such as cars, asuka, falling leaves, etc. on the shooting screen, or when the camera shakes heavily Can occur. Such a miscorresponding point causes an error in subsequent relative orientation, bundle adjustment, and the like, and causes an increase in processing time. Therefore, it is necessary to exclude it as much as possible.

  S23 to S25 are to extract the erroneous corresponding points from the obtained corresponding points and exclude them from the corresponding feature points. The first tracking determination step of S23, the second tracking determination step of S24, Consists of template matching. The first tracking determination unit (S23) estimates coordinate conversion projection parameters between screens (hereinafter simply referred to as “projection parameters”) from the movement of the entire screen from the tracking results. Specifically, this will be described with reference to FIG. 3B. In the second tracking determination step (S24), processing is performed in response to the result of S23. Based on the projection parameters estimated in the first tracking determination step, each corresponding point is evaluated, and the erroneous corresponding point is determined as a feature point. This is excluded from the information memory 9 and will be specifically described with reference to FIG. 3C. In the re-template matching (S25), template matching is performed again on the corresponding points excluding the erroneous corresponding points, the corresponding points are evaluated again based on the result, and inappropriate corresponding points are further excluded (S26). Specific contents will be described with reference to FIG. 3D.

  In S27, these processes are repeated, and when the corresponding point correction of a certain number of frames or more is completed, the next corresponding S30 is output as a temporary corresponding point. The number of frames here is appropriately determined according to the processing capacity and speed of the system.

FIG. 3B shows the processing contents of the first tracking determination step. First, typical processing contents will be described, and then other specific examples will be described.
This process is a process related to two frames, one captured image frame (referred to as frame A) and another captured image frame (referred to as frame B). First, a set of a plurality of points is randomly selected from the candidate corresponding points obtained by template matching in S22. The number of sets is greater than or equal to the number of projection parameters in the next process (S232). Note that the candidate corresponding points to be selected are preferably selected over the entire frame without being localized in a part of the screen in order to reduce errors. Next, the elements (projection parameters) of the projection transformation matrix between the frames A and B are obtained. There are various projective transformations, but affine transformation and Helmat transformation are usually used. Here, an example of affine transformation that is relatively easy to handle both translational transformation and rotational transformation will be described.

If the coordinate values of the corresponding frame are (x ′, y ′), (x ″, y ″), the affine transformation is expressed using the following six projection parameters.
x ′ = ax ″ + by ″ + c (1a)
y ′ = dx ″ + ey ″ + f (1b)
The six projection parameters a to f are obtained from the coordinates of a pair of six or more selected corresponding points. Using this parameter and Equation (1), one of the selected corresponding points is fixed and the other coordinate value (predicted coordinate) is obtained. Next, for each corresponding point, a residual between the obtained predicted coordinates and the actual corresponding point coordinates is obtained (S233). This residual may be, for example, a difference in distance between two coordinate points on the x-axis and the y-axis. If all the corresponding points are recognized correctly, this residual should be very small. Among the six or more pairs of corresponding points selected above, there are some that contain large errors, although not necessarily miscorresponding points or miscorresponding points. If such a corresponding point is selected, a correct projection parameter cannot be obtained. Therefore, the selected corresponding points are evaluated based on the residual. As this embodiment, the following median value (median value) of residuals will be described. As is well known, the median value is the size of the data at the center of a plurality of data sorted in order of size. For example, when there are 5 data, the median value is the value of the third data. The median value can be used to grasp the overall trend of the population. The median value may be sorted by data as an operation, and observation data having a large variance value is ignored, so that there is an advantage that it is difficult to be influenced by data having a large residual. In the present embodiment, the median value is obtained from the residuals of the corresponding points (S234), and it is determined whether this value is a predetermined value (ε1). The square of the median value may be used. If it is larger than the predetermined threshold value (ε1), the selected corresponding point is inappropriate, and the process returns to S231 to select a new corresponding point. The same processing is performed for the new corresponding point, and if the median value is equal to or smaller than the predetermined threshold value (ε1), the projection parameter at that time is obtained (S236). The threshold value at this time is set to 1 pixel (pixel), for example.

  In the present embodiment, the algorithm (LMedS method) that minimizes the median value or the square of the median value has been described. However, the present invention is not limited to this, and the so-called robustness with little influence of errors is described. The same effect can be obtained by using an estimation method (for example, Lanzac method, median estimation method (M estimation method)). Since these estimation methods are well-known, description is abbreviate | omitted here.

  FIG. 3C explains the processing contents of the second tracking determination unit. From the projection parameters obtained in the above-described processing of S23 (see FIG. 3B), corresponding points in the frame B corresponding to the feature points of the frame A are calculated as predicted values, and for all corresponding points, the corresponding points of the corresponding points are calculated. The residual with the coordinate value in frame B is obtained. Next, a threshold value process (S243) is performed for comparing each corresponding point with a value of the residual value larger than a predetermined threshold value, and those larger than the threshold value are excluded as erroneous corresponding points (S244).

  FIG. 3D shows the contents of the re-template matching in S25. Using the projection parameters determined by the first tracking determination unit, estimated coordinates of each corresponding point are determined for the point determined to be an erroneous corresponding point (S237), and template matching is performed again near this coordinate point (S238). As a result, when a corresponding point within a predetermined distance from the estimated coordinates is found, this point is added to the feature point information memory 9 as a corresponding point, and when matching is performed at a point far from the predetermined distance, or a matching point Is not found from the feature point information memory 9 as an inappropriate corresponding point. However, if the corresponding point information is not in the feature point information memory 9, it is left as it is. As described above, the corresponding points that have not been excluded by the second tracking determination unit and the corresponding points newly obtained in the re-template matching (S25) are combined and set as new corresponding points.

  Regarding the above tracking processing, an example of processing a captured image in real time has been described. However, it is also possible to store the captured image in a memory or the like and to perform non-real time processing in which the image data is processed after shooting. Of course. In addition, it is possible to thin out appropriately without processing all captured frames. The thinning screen can be adaptively determined from the moving speed of the photographing apparatus and position data from other means.

[Rejection of miscorresponding points by relative orientation using robust estimation and selection of stereo pairs] Selection of stereo pairs]
FIG. 4 shows specific processing contents. This processing is performed by performing relative orientation processing using robust estimation, thereby removing erroneous corresponding points, and obtaining a pair of frames (stereo pairs) having different parallaxes that can be used for three-dimensional measurement. Corresponding points obtained by tracking processing are performed as input data. First, a plurality of stereo pair candidates from which a certain length of baseline length is obtained are selected (S41). For example, the arrangement of feature points observed on a certain frame is examined on the subsequent frames, and if there is sufficient movement, or if the arrangement is biased to one side of the screen, multiple pairs are selected as stereo pair candidates. . These use the number of frames, the speed, etc., or the locus of the point being tracked.
Next, in the first relative orientation processing unit (S42), a stereo pair is selected at random from the stereo pair candidates, and relative orientation is performed based on a plurality of points corresponding to each other between two captured image frames. Then, the vertical parallax is obtained, robust estimation is performed on the obtained vertical parallax, and an estimated value of the relative orientation parameter is obtained with less influence of a large error.
When removing the miscorresponding point, the second relative orientation processing unit (S43) performs the relative orientation again based on the estimated relative orientation parameters (external orientation elements) estimated by the first relative orientation processing unit. And recalculate the vertical parallax of the feature points, perform threshold processing on the vertical parallax, select a stereo pair that can provide sufficient stereoscopic vision, and support in the stereo pair selected by the second relative orientation processing unit Points are evaluated, and points that are not suitable as corresponding points are excluded as erroneous corresponding points (S43). Next, in S44, a relative orientation (specific details will be described later) is performed using corresponding points other than the erroneous corresponding points, and the selected stereo pair is re-evaluated. If appropriate, it is adopted as a stereo pair. If it is not suitable, the process returns to the first relative orientation processing unit (S42) again, another stereo pair is selected, and the same processing is performed. In this way, an appropriate relative orientation parameter value is estimated by the first relative orientation processing unit, an inappropriate stereo pair is excluded, an erroneous correspondence point is excluded by the second relative orientation processing unit, and a subsequent calculation ( In S44), inappropriate stereo pairs are eliminated. By selecting an appropriate stereo pair through this processing (S41 to S45) and removing the incorrect correspondence, the optimum relative orientation parameters (external orientation elements) and corresponding points at this stage can be obtained.

  The relative orientation is a process of setting the relative position and inclination of the coordinate axes between the stereo pairs in a similar relationship to that at the time of photographing. When the relative orientation is completed, a stereoscopically viewable stereo image can be created. At this time, the optical paths of the same point appearing in the two images between the two images are all on one straight line (a spatial straight line formed by two points is included in the same plane: coplanar condition). Actually, it is off the straight line due to imperfection. This shift is called longitudinal parallax, and is a measure of the relative orientation perfection. In other words, if the coordinate axes and coordinate values are converted between two images so that the vertical parallax is eliminated, the coplanar condition is satisfied, and an appropriate stereo pair is obtained.

A specific mutual orientation procedure will be briefly described. FIG. 7 is an explanatory diagram of relative orientation. In relative orientation, each parameter is obtained by the following coplanar conditional expression.
As shown in FIG. 7, the origin of the model coordinate system (the coordinate system of the object to be imaged) is taken as the left projection center, and the line connecting the right projection centers is taken as the X axis. For the scale, the base line length is taken as the unit length. The parameters to be obtained at this time are the Z-axis rotation angle κ1, the Y-axis rotation angle φ1, the right-side camera O2 the Z-axis rotation angle κ2, the Y-axis rotation angle φ2, and the X-axis rotation angle ω2. 5 rotation angles. In this case, since the rotation angle ω1 of the X axis of the left camera O1 is 0, there is no need to consider it. Under such conditions, the coplanar conditional expression of Expression 2 becomes Expression 3, and each parameter can be obtained by solving this expression.

Here, the following relational expression for coordinate transformation is established between the model coordinate system XYZ and the camera coordinate system xyz.

Using these formulas, for example, unknown parameters are obtained by the following procedure.
(1) The initial approximate value is normally 0.
(2) The coplanar conditional expression (2) is Taylor-expanded around the approximate value, and the value of the differential coefficient when linearized is obtained by the expressions (4) and (5), and the observation equation is established.
(3) Apply an estimation method to find the correction amount for the approximate value.
(4) Correct the approximate value.
(5) Using the corrected approximate value, the operations (2) to (5) are repeated until convergence.

  Next, the contents of the first relative orientation processing unit (S42) will be described in detail with reference to FIG. In this process, the relative orientation parameters are estimated by performing relative orientation between the selected stereo pairs of image frames, and the relative orientation is performed by the second relative orientation processing unit based on the estimated parameters. (S430), and detection / exclusion of miscorresponding points. Then, relative orientation is performed again based on the result (S441), and whether or not the stereo pair is optimal is evaluated and selected. The erroneous corresponding point has already been excluded in the tracking process (S20) described above, but it is a two-dimensional process to the last, and since there is an unsuitable corresponding point is not excluded, the relative orientation process is performed, and the vertical parallax is reduced. By evaluating, a pseudo three-dimensional evaluation is performed, and only appropriate corresponding points are left.

  First, one stereo pair is randomly selected from the stereo pair candidates (S41) (S41a). Next, a plurality of corresponding points are selected at random. The number of corresponding points is 5 or more (S422). Next, relative orientation is performed using the corresponding points of 5 points or more (S423). As a result of the relative orientation, a relative orientation parameter using the corresponding points is obtained, and for each corresponding point, a vertical parallax (an index of a relative orientation error) ) Can be calculated. The size of the vertical parallax serves as an evaluation scale for determining whether the corresponding points are appropriate, and the size should be as small as possible. Here, in order to perform this evaluation, a median value of vertical parallax is obtained for each corresponding point (S424), and the value is compared with a preset threshold value ε2 (second predetermined value) (S425). If it is larger than the predetermined value, the process returns to S41a to set a new corresponding point. The threshold value at this time is set to about 1 pixel (1 pixel), for example. When the value is equal to or smaller than the predetermined value, it is determined that the optimum relative orientation parameter at this stage has been estimated, and the process proceeds to the second mutual evaluation processing unit.

  Here, the case where the algorithm LMedS method for obtaining the median value and minimizing the median value is used has been described. However, the present invention is not limited to this, and a so-called robust estimation method with little influence of error variation can be widely used. Is clear.

  Next, processing contents of the second relative orientation processing unit will be described. Here, based on the result of S42, relative orientation is performed from the estimated relative orientation parameters (S430), and the vertical parallax of each corresponding point in the frame of the stereo pair is calculated (S431). The corresponding points may be all points or a part, but most of the points in the screen are desirable. Threshold processing is performed on these vertical parallaxes (S432), and corresponding points equal to or higher than the threshold value (ε3) are excluded from the feature point information memory 9 as erroneous corresponding points. The threshold value (ε3) at this time is set to a value where the vertical parallax is about 0.5 to 1 pixel, for example. This threshold value may be changed depending on the desired accuracy and the object. This eliminates an inappropriate corresponding point on the stereo pair.

  Next, in order to evaluate whether the stereo pair is appropriate, the relative orientation is performed again only with the corresponding points excluding the erroneous corresponding points (S441), the relative orientation parameters are obtained, and the vertical parallax of each corresponding point is obtained. (S442). The median value of the vertical parallax is compared with a predetermined value ε4 (fourth predetermined value) (S443). When the median value is equal to or higher than the predetermined value, the stereo pair is inappropriate, so the process returns to S41a and a new stereo pair is selected. To do. When it is smaller than the predetermined value, it is determined as an appropriate stereo pair candidate. Here, the “candidate” is used because the stereo pair may be unsuitable in the large error removal process by the next bundle.

  As the robust estimation method, RANSAC estimation, M estimation method, or a modified version thereof can be used. These are the same in that the evaluation method and its evaluation function are different from LMedS and are not easily affected by large observation errors.

  Note that it is also possible to remove only the erroneous corresponding points without selecting the stereo pair. In that case, S41a and S44 need not be performed. Alternatively, it is not necessary to perform only the processing of S43 as appropriate frame selection. In this case, the processes of 41a and S42 are performed as the first relative orientation processing unit, and only the process of S44 is performed as the second relative orientation processing unit. That is, the relative orientation parameters are estimated in S42, and the suitability of the optimum frame is determined in S44. These are appropriately configured as the performance of the entire system, for example, depending on the speed and tracking unit's ability to remove incorrect correspondence, the target landscape, the measurement target, and the like.

[Bundle adjustment 1]
Next, paying attention to the movement of corresponding points between a plurality of frames, the corresponding points and frames that move abnormally are excluded. Specifically, corresponding points and frame images are evaluated for a plurality of frame images by bundle adjustment.

FIG. 5 shows the processing contents. S40 / S50 is the process of removing the miscorresponding points and selecting the optimum stereo pair by the relative orientation process described with reference to FIGS. In this process, these processes are performed in a stage prior to this process, and are performed in response to this process. A plurality of frames are selected for the selected frame (S61). Next, connection orientation is performed for these frames. Connection orientation refers to the coordinate system (model coordinate system) that is unique to the stereo model generated for each stereo pair and is related to neighboring models, and the inclination and scale between the models are unified to create a unified coordinate system (course A process described in the coordinate system. Specifically, in adjacent image coordinates, for example, a point in the right model is (X jr , Y jr , Z jr ), and a corresponding point in the left model is (X jl , Y jl , Z jl ), When the coordinates of the camera 2 are (X 0 , Y 0 , Z 0 ), the following ΔX j , ΔY j , ΔZ j , ΔD j are calculated, and if ΔZ j and ΔD j are sufficiently small (for example, 1 / 2000 or less), it is determined that connection orientation has been performed normally.
ΔX j = (X jr −X jl ) / (Z 0 −Z jl ) (Formula 6a)
ΔY j = (Y jr −Y jl ) / (Z 0 −Z jl ) (Formula 6b)
ΔZ j = (Z jr −Z jl ) / (Z 0 −Z jl ) (Formula 6c)
ΔD j = (ΔX j 2 + ΔY j 2 ) 1/2 (Formula 6d)

Next, bundle adjustment is performed based on each selected frame image (S63). The bundle adjustment is to obtain an external orientation element of each frame image collectively by establishing an equation relating a plurality of (three or more) frame images and three-dimensional coordinates.
Specifically, the following collinear condition (Expression 7), which is a basic expression for bundle adjustment that the projection center, the photographic image, and the object on the ground are in a straight line, is used. Six external orientation elements X 0 , Y 0 , Z 0 , ω, φ, κ included in Expression 7 are obtained. That is, these six external orientation elements are calculated by the successive approximation method from the image coordinates corresponding to the object coordinates of three or more reference points.
here,
c: Screen distance (focal length), x, y: Image coordinates
X, Y, Z: Target space coordinates (reference point, unknown point)
X 0 , Y 0 , Z 0 : Shooting position a ij of camera 2: Tilt of camera 2 (element of 3 × 3 rotation matrix)
Δx, Δy: Camera internal orientation correction terms.
Specifically, the approximate value of the unknown variable is given, equation 7 is tailored around the approximate value to be linearized, the correction value is obtained by the least square method, the approximate value is corrected, and the same operation is repeated to converge. Iterative approximation is used to find Thus, these six external orientation elements are obtained. Further, the residual of the image coordinates of each corresponding point is obtained, the residual is compared with a predetermined threshold value ε5 (fifth predetermined value), the corresponding point is excluded for those having the threshold value ε5 or more (S64), and S40 is again performed. Returning to the process of S50, the optimum stereo pair is selected (specifically, the miscorresponding point is removed by the relative orientation process and the optimum frame is selected). If the residuals of all corresponding points are smaller than the predetermined value, the process is terminated (S64), the three-dimensional coordinates and the external orientation elements are output (S65), and the process is terminated.

[Bundle adjustment 2]
FIG. 6 shows an evaluation method by bundle adjustment according to another method. By this method, the final three-dimensional coordinates and external orientation elements are obtained by removing the error of the coordinate position data of each feature point and the error of each frame photographing position.
Steps S20 / S30 to S62 ′ are the same as in FIG. The connection orientation of S62 ′ is the same process as the connection orientation S62 of FIG.

  Next, the first bundle adjustment processing unit (S67) performs robust estimation on the obtained coordinate position data of the feature points or the residual of the shooting position of each frame, obtains the estimated data, selects The removal processing unit (S68) determines those threshold values based on the estimation data estimated by the first bundle adjustment processing unit, and removes miscorresponding points or selects captured image frames based on the determined threshold values. Then, the final bundle adjustment is performed (S682), and the three-dimensional coordinates and the external orientation elements of each frame are obtained (S69).

  The first bundle adjustment processing unit (S67) will be described. A selection candidate for the frame to be used in S671 is selected, and a plurality (three or more) of frames are selected at random (S672). Usually, the number of videos or moving images is quite large, so you can use all of them, but it is better to select the candidates to use in consideration of speed and hardware load. For example, sampling is performed at a fixed frame interval, or the one selected in the relative orientation process and its periphery are selected. Next, corresponding points (feature points) are selected at random (S673). The bundle adjustment calculation described above is performed (S674), and the corresponding point coordinate estimated position data and the estimated shooting position (external orientation element) of each frame are calculated (these are obtained simultaneously). Next, a robust estimation criterion is calculated. That is, a coordinate residual other than the selected point is calculated (S675), and its median value is obtained and used as a robust estimation criterion. The evaluation value in this case is the residual of the estimated point. For example, it is set to about 1 pixel (pixel) (S676). This method is a so-called robust estimation method, specifically the LMedS method described above. That is, the residual of the image coordinates of each corresponding point is obtained, and if there is a residual whose median value is larger than a predetermined threshold ε6 (sixth predetermined value), the process returns to S673, and a new corresponding point is obtained. And repeat the same process. If the median value is equal to or smaller than the threshold ε6, the process proceeds to the selection / removal processing unit (step) (S68).

In the selection / removal processing unit (step) (S68), the estimated coordinate position data obtained in the first bundle adjustment processing unit (step) (S67) or the magnitude of the photographing position residual is compared with the threshold ε7. The coordinate position data having a larger error than that is excluded as a miscorresponding point (S68). A frame corresponding to a shooting position having a large large error is excluded as an inappropriate image frame. The threshold value ε7 is, for example, 0.5 pixel if the accuracy is to be improved, and about 3 pixels if the accuracy is not so high.
Next, the final bundle adjustment is performed once again excluding these miscorresponding points (S682), and the values calculated in the final bundle adjustment of S682 are output as three-dimensional coordinates and external orientation elements of each frame (S69). ).

  In the above description, the selection of the frame (imaging position) and the removal of the large error point (coordinate position data) are performed based on the robust estimation criterion, but either one or both may be performed simultaneously. For example, if processing is heavy, robust estimation of frames may be excluded.

[Second Embodiment]
In recent years, the use of GPS has become easier, and the accuracy has been improved by the development of relative positioning methods such as DGPS and interference positioning methods such as RTK-GPS even in consumer open systems. It is on the order of several centimeters. In this embodiment, the position measurement data obtained by GPS is used in a complementary manner to improve the overall accuracy and shorten the measurement time.

  Specifically, there is one that applies GPS data to tracking. This is to associate the measurement data in GPS with the frame of the position of the camera 2 corresponding to each frame, and specifically, if the measurement output value and the captured image frame are synchronized in GPS, the GPS When the measurement frames are output and the shooting frames are associated and are not synchronized or cannot be synchronized, use the GPS and the time stamp of the camera to associate the frames with the closest time . Alternatively, another timer may be prepared in hardware. It is ideal that it is properly synchronized, but an approximate value may be used in the present invention.

  GPS data can also be applied to the selection of a stereo pair. That is, since each frame and the position of the camera 2 are associated with each other, it is possible to select a frame having an appropriate baseline length. Moreover, even if the photographing camera and the GPS measurement output value are not properly synchronized, the robust estimation selection of the frame of the present invention can be applied and processed. FIG. 8 shows a specific application example. FIG. 8 is to select a stereo pair having a sufficient base line length from the position data of the camera 2 by GPS as in S41a ′ of FIG. 8 instead of randomly selecting the portion of S41a in FIG. . Here, as described above, the base line length may be an approximate value even if it is not properly synchronized and associated. With this configuration, useless frames are not selected. Further, an appropriate frame can be selected without extra hardware for synchronization.

  Furthermore, in bundle adjustment, camera position data by GPS is used as an initial value for bundle adjustment. In this way, since the calculation can be started from a value close to the actual value, there is an advantage that the convergence time can be remarkably shortened.

  As described above, according to the position measuring apparatus and the position measuring method of the present invention, the estimation calculation used in the surveying method by digital data processing of a plurality of photographs is not easily affected by a large observation error as the first step. Since the robust estimation method is applied, the miscorresponding point that is a large error factor and the inappropriate frame are removed based on the estimation result, and the final estimation calculation that is the second stage is performed after removing the error factor. The convergence time is short as a whole, and highly accurate measurement results can be obtained.

FIG. 1 is a diagram showing a hardware and software module configuration of the present invention. FIG. 2 is an overall flowchart of the image processing method according to the first embodiment. FIG. 3A shows the processing contents of the tracking processing unit. FIG. 3B is a diagram illustrating details of the first tracking determination unit (process) of FIG. 3A. FIG. 3C is a diagram illustrating details of the second tracking determination unit (process) of FIG. 3A. FIG. 3D is a diagram illustrating details of the re-template matching unit (process) in FIG. 3A. FIG. 4 is a diagram showing details of the relative orientation unit (process). FIG. 5 is a diagram showing details of the bundle adjustment processing unit (process). FIG. 6 is a diagram showing details of a bundle adjustment processing unit (process) according to another configuration. FIG. 7 is a diagram showing the principle of relative orientation. FIG. 8 is an example of a relative orientation processing unit (process) when GPS data is used. FIG. 9 is a diagram showing the concept of the present invention.

Explanation of symbols

1 Operation unit 2 Captured image acquisition unit (camera)
3 Feature Extraction Unit 4 Tracking Unit 4A Feature Point Tracking Unit 4B First Tracking Judgment Unit 4C Second Tracking Judgment Unit 4D Retemplate Matching Unit 5 Mutual Orientation Unit 5A First Mutual Orientation Processing Unit 5B Second Mutual Orientation Processing Unit 5C third relative orientation processing unit 6 bundle adjustment unit 6A first bundle adjustment processing unit 6B selection / removal processing unit 6C final bundle adjustment unit 7 position measurement unit 8 display unit 9 feature point information memory 10 moving image memory 11 external Output unit 100 Position measuring device 101a, 101b Feature point

Claims (7)

  1. Feature extraction that extracts a plurality of feature points of a photographic object in an image from a photographic image composed of a plurality of frames obtained from a photographic image acquisition means whose position relative to the photographic object changes with time Part;
    A feature point tracking unit that searches corresponding points corresponding to the feature points of the plurality of captured images and tracks the feature points;
    Based on the positional relationship between the feature point of one image and the feature point on the other image from the tracking process or result of the feature point tracking unit, it was obtained using the projection parameters between the one image and the other image A first tracking determination unit that performs robust estimation on the residual of the coordinate value and obtains an estimated value of the projection parameter;
    Using the projection parameter estimation value obtained by the first tracking determination unit, a residual of the coordinate value from the predicted value of the corresponding point of each feature point is obtained, threshold processing is performed, and each feature point is erroneously detected. A second tracking determination unit that determines whether or not the corresponding point is a corresponding point and excludes the erroneous corresponding point from the feature point;
    Using the projection parameter estimated value obtained by the first tracking determination unit, the coordinate value of the estimated position of the erroneous corresponding point is obtained, template matching is performed in the vicinity of the position, and the coordinate value of the appropriate corresponding point A re-template matching unit for
    Based on the coordinate value of each corresponding point other than the erroneous corresponding point obtained by the second tracking determination unit and the coordinate value of the appropriate corresponding point obtained by the template matching unit, A position measurement unit that measures a photographing position or a photographing posture of the photographed image acquisition unit.
  2. The robust estimation is any one of the LMedS method, the median estimation (M estimation) method, and the RANSAC method;
    The position measuring device according to claim 1.
  3. The first tracking determination unit obtains a projection parameter between the shooting screens from a plurality of partial feature points selected from the plurality of acquired feature points, and from the projection parameters, a feature point other than the partial feature points is obtained. Calculating the coordinates of each corresponding point, and obtaining a residual of the coordinate value of the feature point based on the coordinate value;
    The position measuring device according to claim 1 or 2.
  4. In the first tracking determination unit, the projection parameter is an affine transformation parameter;
    The position measuring device according to claim 1.
  5. The robust estimation in the first tracking determination unit employs the LMedS method, and based on the projection parameters obtained from the partial corresponding points, the median value of the magnitude of the difference between the coordinate values is calculated for each corresponding point. Determining, performing the same process on the other combinations of the partial corresponding points, selecting the combination of the partial corresponding points that minimizes the calculated median value, and determining the projection parameter;
    The position measuring device according to claim 3 or 4.
  6. A GPS position data acquisition unit that obtains shooting position data from GPS data, and associates the acquired position data with the shooting position data of each frame obtained by the position measurement unit;
    The position measuring device according to claim 1.
  7. Feature extraction that extracts a plurality of feature points of a photographic object in an image from a photographic image composed of a plurality of frames obtained from a photographic image acquisition means whose position relative to the photographic object changes with time Process and;
    A feature point tracking step of searching for a corresponding point corresponding to the feature point for a plurality of the captured images and tracking the feature point;
    Based on the tracking process or result of the feature point tracking process, based on the positional relationship between the feature point of one image and the feature point on the other image, using the projection parameters between the one image and the other image A first tracking determination step of performing robust estimation on the residual of the coordinate values obtained and obtaining an estimated value of the projection parameter;
    Using the projection parameter estimation value obtained in the first tracking determination step, a residual of the coordinate value from the predicted value of the corresponding point of each feature point is obtained, threshold processing is performed, and each feature point is erroneously detected. A second tracking determination step of determining whether the point is a corresponding point and excluding the erroneous corresponding point from the feature point;
    Using the projection parameter estimation value obtained in the first tracking determination step, the coordinate value of the estimated position with respect to the erroneous corresponding point is obtained, template matching is performed near the position, and the coordinate value of the appropriate corresponding point Re-template matching process to find out;
    Based on the coordinate value of each corresponding point other than the erroneous corresponding point obtained by the second tracking determination unit and the coordinate value of the appropriate corresponding point obtained in the template matching step, A position measurement step of measuring a shooting position or a shooting posture of the shot image acquisition means.
JP2007179036A 2007-07-06 2007-07-06 Position measuring apparatus and position measuring method Active JP5084374B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2007179036A JP5084374B2 (en) 2007-07-06 2007-07-06 Position measuring apparatus and position measuring method

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2007179036A JP5084374B2 (en) 2007-07-06 2007-07-06 Position measuring apparatus and position measuring method
US12/166,660 US7844077B2 (en) 2007-07-06 2008-07-02 Location measuring device and method
US12/166,654 US7787689B2 (en) 2007-07-06 2008-07-02 Location measuring device and method

Publications (2)

Publication Number Publication Date
JP2009014628A JP2009014628A (en) 2009-01-22
JP5084374B2 true JP5084374B2 (en) 2012-11-28

Family

ID=40355694

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2007179036A Active JP5084374B2 (en) 2007-07-06 2007-07-06 Position measuring apparatus and position measuring method

Country Status (1)

Country Link
JP (1) JP5084374B2 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102414719B (en) * 2009-07-22 2014-10-15 欧姆龙株式会社 Surveillance camera terminal
US8817071B2 (en) * 2009-11-17 2014-08-26 Seiko Epson Corporation Context constrained novel view interpolation
US8553982B2 (en) * 2009-12-23 2013-10-08 Intel Corporation Model-based play field registration
JP5573606B2 (en) * 2010-11-04 2014-08-20 アイシン・エィ・ダブリュ株式会社 Reference pattern information generating device, method, program, and general vehicle position specifying device
JP5582009B2 (en) * 2010-12-10 2014-09-03 富士通株式会社 3D position measurement system, 3D position measurement apparatus, and 3D position measurement method
JP2012202942A (en) * 2011-03-28 2012-10-22 Casio Comput Co Ltd Three-dimensional modeling device, three-dimensional modeling method, and program
JP6188387B2 (en) * 2012-04-17 2017-08-30 パナソニック株式会社 Parallax calculation apparatus and parallax calculation method
WO2014141522A1 (en) 2013-03-12 2014-09-18 富士フイルム株式会社 Image assessment device, capturing device, 3d measuring device, image assessment method, and program
KR101852085B1 (en) 2016-08-16 2018-04-26 한국과학기술원 Depth map acquisition device and depth map acquisition method
WO2019058487A1 (en) * 2017-09-21 2019-03-28 オリンパス株式会社 Three-dimensional reconstructed image processing device, three-dimensional reconstructed image processing method, and computer-readable storage medium having three-dimensional reconstructed image processing program stored thereon

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003115052A (en) * 2001-10-09 2003-04-18 National Institute Of Advanced Industrial & Technology Image processing method and image processor
JP2005332177A (en) * 2004-05-19 2005-12-02 Fukui Computer Kk Three-dimensional information processor, three-dimensional information processing method, and three-dimensional information processing program
US7860276B2 (en) * 2005-12-08 2010-12-28 Topcon Corporation Image processing device and method

Also Published As

Publication number Publication date
JP2009014628A (en) 2009-01-22

Similar Documents

Publication Publication Date Title
US10495461B2 (en) Surveying system
Mur-Artal et al. ORB-SLAM: a versatile and accurate monocular SLAM system
Tanskanen et al. Live metric 3d reconstruction on mobile phones
Ventura et al. Global localization from monocular slam on a mobile phone
US20160300383A1 (en) Human body three-dimensional imaging method and system
EP2615580B1 (en) Automatic scene calibration
US9020187B2 (en) Planar mapping and tracking for mobile devices
US9171225B2 (en) Device, method, and recording medium for detecting and removing mistracked points in visual odometry systems
US9911226B2 (en) Method for cleaning or processing a room by means of an autonomously mobile device
US20170277197A1 (en) Autonomous Navigation using Visual Odometry
EP2660777B1 (en) Image registration of multimodal data using 3D geoarcs
Konolige et al. Large-scale visual odometry for rough terrain
Alcantarilla et al. On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments
US20170070724A9 (en) Camera pose estimation apparatus and method for augmented reality imaging
JP5960595B2 (en) Image processing apparatus, image processing method, program, and photographing apparatus
US8818132B2 (en) Camera calibration with lens distortion from low-rank textures
Gracias et al. Underwater video mosaics as visual navigation maps
Hartley Self-calibration of stationary cameras
Ventura et al. Wide-area scene mapping for mobile visual tracking
JP2016521892A (en) Interactive and automatic 3D object scanning method for database creation
US9969337B2 (en) Methods and systems for mobile-agent navigation
US8588471B2 (en) Method and device of mapping and localization method using the same
US9224208B2 (en) Image-based surface tracking
US20150332464A1 (en) Methods for automatic registration of 3d image data
JP3833786B2 (en) 3D self-position recognition device for moving objects

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20100507

TRDD Decision of grant or rejection written
A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120802

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20120807

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20120904

R150 Certificate of patent or registration of utility model

Ref document number: 5084374

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20150914

Year of fee payment: 3

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250