JP2008298589A - Device and method for detecting positions - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for accurately detecting positions of objects at various distances at close to long range without being affected by the change in distortion aberration due to distance by using a single model. <P>SOLUTION: The position detecting device is equipped with an optical system 22 and an image processing section 3 for detecting the position of an object P shot by image-pickup means 2a and 2b by applying an intersection displacement model in which the intersection of a light flux coming into the optical system 22 and the optical axis of the optical system 22 moves along the extension of the optical axis according to measured values pertaining to the image focus location of the object P obtained by a plurality of imaging means 2a and 2b to pick up the image of the object P on the imaging surface 25 or position-related information pertaining to the object P with respect to the optical system 22. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a position detection device and a position detection method.

  2. Description of the Related Art Conventionally, a position detection device and a position detection method are known in which an object is imaged using a plurality of imaging devices to obtain a three-dimensional coordinate (position) of the object.

  However, various types of aberrations such as chromatic aberration and distortion are included in an image obtained by imaging an object with an imaging apparatus using an optical system including lenses. Among these, distortion is the most serious problem when the imaging device is used for measurement, such as when detecting the position of an object using an imaging device. If it is included, the position of the object cannot be detected accurately.

  Conventionally, as a technique for correcting this distortion, a technique is known in which distortion is expressed as a function, and an input image is corrected based on a functional expression, in order to improve correction accuracy. It is necessary to improve the approximation accuracy of this function.

  In this regard, for example, a method is proposed that employs a method of increasing the polynomial term until the accuracy of the function reaches the target performance (see, for example, Patent Document 1). In addition, a method for optimizing approximation by a function by using a special function and improving the accuracy of correction has been proposed (see, for example, Patent Document 2).

However, it is known that the distortion aberration changes according to the distance from the object to the imaging plane. For this reason, even if high-accuracy correction can be performed at a certain distance, the correction accuracy may decrease due to a change in the distance from the object to the imaging plane.
In order to cope with this, a method has been proposed in which a plurality of distortion correction curves are created according to the distance from the object to the imaging plane, and the measurement values are corrected based on the obtained distortion aberration correction curves. (For example, refer to Patent Document 3).

  As described above, conventionally, as a technique for correcting distortion, a technique for accurately clarifying distortion occurring in a captured image and a technique for changing the correspondence of distortion according to the distance of an object are known. With these two, the accuracy of correction is improved, and the accuracy of position detection of the object is aimed at.

As a model for calculating the distance of an object in a function used for distortion correction, there is always a light beam from the object as disclosed in Patent Document 4, Patent Document 5, and the like. A so-called pinhole model of passing through one point was applied.
JP-A-6-217142 JP 2001-133223 A JP 2001-133225 A JP 2005-91173 A JP 2003-304561 A

However, in the technique described in Patent Document 3 and the like, a technique is adopted in which a plurality of distortion aberration correction curves are provided for distortion aberrations that differ for each distance, and a processing apparatus that performs correction has a large memory capacity. Besides being necessary, there is a problem that internal control becomes complicated.
Further, for the portion where the correction curve is not created, it is necessary to devise such as processing by interpolating this portion, which complicates the control and causes a poor correction accuracy.

Further, in the technique described in Patent Document 3 or the like, for example, when an object is fixed and imaged on a table, and the distance from the table to the imaging device is known in advance, the distance is set in advance. Since a corresponding correction curve for distortion can be created accurately, high-accuracy correction can be performed by correcting based on this correction curve. Even when the table moves up and down and the distance from the table to the imaging device can be changed, if the distance for each position is known, a correction curve corresponding to each position can be similarly created in advance. it can.
However, in the case of general imaging, there are many cases where the distance from the object to the imaging device is not known, or there are many objects with different distances mixed in one screen. However, there is a problem that the technique described in Patent Document 3 cannot be used.

  Therefore, the present invention has been made to solve the above-described problems, and it is possible to easily cope with a change in distortion due to distance with a single model, and at various distances from a short distance to a long distance. It is an object of the present invention to provide a position detection device and a position detection method that can detect the position of a certain object with high accuracy.

In order to solve the above problem, the invention according to claim 1 is a position detection device,
The optical system based on actual measurement values on the imaging position of an object on the imaging plane obtained by a plurality of imaging means provided with an optical system and imaging the object or position related information on the position of the object with respect to the optical system Position detection for detecting the position of the object imaged by the imaging means by applying an intersection displacement model in which the intersection of the light beam incident on the optical system and the optical axis of the optical system is back and forth in the extending direction of the optical axis It is characterized by having means.

The invention according to claim 2 is the position detection apparatus according to claim 1,
The position detecting unit includes a normalizing unit that performs normalization conversion processing to obtain normalized coordinates by normalizing camera coordinates, and a shift that performs shift correction processing of the plurality of imaging units after being normalized by the normalizing unit. Correction means,
The intersection displacement model is applied in the deviation correction processing in the deviation correction means.

The invention according to claim 3 is the position detection device according to claim 1 or 2,
Coefficient calculating means for calculating a coefficient in the intersection displacement model is further provided.

According to a fourth aspect of the present invention, in the position detection method,
The optical system based on actual measurement values on the imaging position of an object on the imaging plane obtained by a plurality of imaging means provided with an optical system and imaging the object or position related information on the position of the object with respect to the optical system Position detection for detecting the position of the object imaged by the imaging means by applying an intersection displacement model in which the intersection of the light beam incident on the optical system and the optical axis of the optical system is back and forth in the extending direction of the optical axis It has the process.

The invention according to claim 5 is the position detection method according to claim 4,
The position detection step includes a normalization step of normalizing a camera coordinate to obtain a normalized coordinate, and a shift for performing a shift correction process of the plurality of imaging units after being normalized by the normalization step A correction step, wherein the intersection displacement model is applied in the deviation correction step.

The invention according to claim 6 is the position detection method according to claim 4 or 5,
The method further includes a coefficient calculation step of calculating a coefficient in the intersection displacement model.

  According to the first and fourth aspects of the invention, when detecting the position of the object, the intersection of the light beam incident on the optical system and the optical axis of the optical system moves back and forth in the extending direction of the optical axis. Since the intersection displacement model is applied, even if the distance from the object to the imaging device is not known, or when multiple objects with different distances are mixed in one screen, it can be handled with one model. The position detection can be easily and highly accurately performed.

  According to the second and fifth aspects of the invention, in the step of performing the deviation correction process, the step of detecting the position of the object is divided into the step of performing the normalization process and the step of performing the deviation correction process. Since the intersection displacement model is applied, there is an effect that a necessary minimum processing load can be achieved.

  According to the third and sixth aspects of the present invention, since the coefficient in the intersection displacement model is calculated and applied to detect the position of the object, the distance from the object to the imaging device is not known. In addition, even when a plurality of objects having different distances are mixed in one screen, there is an effect that the position of the object can be detected easily and with high accuracy.

  The best mode for carrying out the present invention will be described below with reference to the drawings. However, the scope of the present invention is not limited to the following embodiments and illustrated examples.

[First Embodiment]
A first embodiment of the position detection device will be described with reference to FIGS. 1 to 9.
As illustrated in FIG. 1, the position detection device 1 according to the present embodiment performs image processing on two imaging devices 2 a and 2 b that capture an object (subject) P and images acquired by these imaging devices 2 a and 2 b. And an image processing unit 3 that corrects distortion aberration of the image and calculates a three-dimensional position of the object (subject).

  Each of the imaging devices 2a and 2b includes an imaging system (photoelectric conversion elements) 21a and 21b and an optical system 22a that forms a subject light image on an imaging surface 25 (see FIGS. 2 and 3) of the imaging elements 21a and 21b. 22b. The image sensors 21a and 21b are image sensors such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal-Oxide Semiconductor), for example. And photoelectrically convert the subject light image to the image processing unit 3 as an image signal. The optical systems 22a and 22b may be configured with one lens or may be configured with a plurality of lenses.

  The image processing unit 3 includes a processing device such as a CPU (Central Processing Unit), a system program, a calibration program for performing calibration processing to be described later, and a position detection program for performing position detection processing for detecting the position of an object. Is a computer configured with a storage unit such as a ROM (Read Only Memory) for storing various control programs, a RAM (Random Access Memory) for temporarily storing various data (all not shown), etc. A calibration process for correcting distortion of an image and a position detection process for detecting the position of an object are performed.

Here, the distortion will be described with reference to FIGS. 2 and 3, the vertical axis in FIGS. 2 and 3 of the imaging surface 25 of the imaging element 21 on which an image is formed is the x axis and the horizontal axis is the y axis. FIG. 2 shows the case of a conventional pinhole model in which the light beam from the object P always passes through a certain point (in the case of FIG. 2, the center of the optical system).
2 and 3, the coordinates (ideal coordinates) of the imaging position when distortion is not generated are (x ₁ , y ₁ ), and the coordinates of the actual imaging position when distortion is occurring ( When the real coordinates are (x ₂ , y ₂ ) and the point where the optical axis of the optical system 22 intersects the image plane 25 is the center (x ₀ , y ₀ ), this center (x ₀ , y ₀ ) The distance from the center (x ₀ , y ₁ ) to the coordinates (x ₁ , y ₁ ) is “r _ideal ”, and the distance from the center (x ₀ , y ₀ ) to the coordinates (x ₂ , y ₂ ) (actual image height) ) As “r _real ”. The distortion aberration of an image obtained when the object P is photographed can be expressed as a ratio between an _ideal image height (r _ideal ) and an actual image height (r _real ), and is calculated by the following equation (1). can do.
Distortion aberration = (r _ideal −r _real ) / r _ideal × 100 (%) (1)

In the case of using the pinhole model shown in FIG. 2, the ideal coordinates (x ₁ , y ₁ ) in this case can be expressed as follows. In the following, X is the distance in the X-axis direction of the object P, Y is the distance in the Y-axis direction of the object P, and Z is the distance from the object P to the center of the optical system.
x ₁ = f (X / Z), y ₁ = f (Y / Z)
Further, when the angle of light incident on the optical system 22 from the object P is θ, the distance (ideal image height) from the center (x ₀ , y ₀ ) to the ideal coordinates (x ₁ , y ₁ ) (r _ideal ) Can be expressed as in the following formula (2).
r = √ (x ² + y ² ) = ftanθ (2)

Further, the distance R from the object P to the optical axis can be expressed as the following formula (3).
R = √ (X ² + Y ² ) (3)

For this reason, ideal coordinates (x ₁ , y ₁ ) can be expressed as follows.
x ₁ = r (X / R), y ₁ = r (Y / R)

In contrast, in the present embodiment, instead of the pinhole model, as shown in FIG. 3, as shown in FIG. 3, measured values relating to the imaging position on the imaging surface 25 of the object P obtained through the optical system 22 or the optical system 22. Based on the position-related information regarding the position of the object P with respect to, an intersection displacement model is applied in which the intersection of the light beam incident on the optical system 22 and the optical axis of the optical system 22 is back and forth in the extending direction of the optical axis. That is, as shown in FIG. 3, the light beam incident on the optical system 22 intersects the optical axis of the optical system 22 at a position shifted from the center of the optical system 22 by k _{1 in} the extending direction of the optical axis.
Here, the actual measurement value relating to the imaging position of the object P on the imaging surface 25 is the coordinates of the actual imaging position of the object P obtained through the optical system 22 (actual coordinates: x ₂ , y _2). ) Or a distance from the point (center: x ₀ , y ₀ ) where the optical axis of the optical system 22 intersects the imaging plane 25 to the coordinates (x ₂ , y ₂ ) (real image height: r _real ) The position-related information regarding the position of the object P with respect to the system 22 includes the X and Y coordinates of the object P, the distance Z from the object P to the optical system center, the distance f from the optical system center to the imaging plane 25, the object The angle θ or the like at which the light beam from the object P intersects the optical axis of the optical system 22.
In the actual optical system 22, it is difficult to configure all the light beams incident on the optical system 22 so as to pass through the center of the optical system 22, and the light beams incident on the optical system 22 also in correcting distortion. It is practical to apply an intersection displacement model in which the intersection of the optical system 22 and the optical axis of the optical system 22 is back and forth.

When such an intersection displacement model is used, the ideal coordinates (x ₁ , y ₁ ) in this case can be expressed as follows.
x ₁ = ((f + f (r)) / (Z−f (r))) X, y ₁ = ((f + f (r)) / (Z−f (r))) Y
Here, f (r) is a function having a predetermined coefficient. In the present embodiment, the function f (r) in the intersection displacement model is a function (“k ₁ r”) that changes linearly according to the image height r (imaging position x, y) as described later. ) Will be described as an example. In addition, _{k 1} is a coefficient.

In the present embodiment, the image processing unit 3 acquires an ideal value (ideal coordinates, ideal image height) using the intersection displacement model, and assumes an assumed value (imaging plane of the object P) corresponding to the ideal value. 25), a value including distortion aberration with respect to the imaging position on 25).
Then, the image processing unit 3 performs a calibration process for calculating and specifying a coefficient k ₁ in the intersection displacement model, a coefficient (for example, α, β, γ) and the like in a later-described distortion aberration model from the actually measured value and the assumed value, The result is stored in a storage unit such as a RAM. Further, a correction process is performed for converting the actual value so as to approach the ideal value from the result of the calibration.

Specifically, the image processing unit 3 compares the actual measurement value (actual coordinates, actual image height) and the assumed value to determine whether the actual measurement value and the assumed value are within a predetermined error range. To do. In addition, a threshold value for determining whether or not it is within the predetermined error range is stored in advance in a storage unit such as a ROM.
When the image processing unit 3 determines that the actual measurement value and the assumed value are within the predetermined error range, the image processing unit 3 derives the coefficient k ₁ and the assumed value in the intersection displacement model provisionally determined when the ideal value is derived. Therefore, a calibration process is performed in which the coefficient in the temporarily determined distortion aberration model is identified as an appropriate coefficient and stored in the storage unit. The image processing unit 3 in this way, in the course of calibration, calculates the coefficient k ₁ at the intersection displacement model, which identifies the image processing section 3 functions as a coefficient calculating means.
Then, the image processing unit 3 corrects the image so as to approximate the actual measurement value to the ideal value corresponding to the assumed value based on the calibration result.

  Furthermore, the image processing unit 3 applies the intersection displacement model to detect the position of the object P imaged by the imaging means 2a and 2b (M (X, Y, Z) in FIG. 4). Functions as position detection means.

In the present embodiment, the position detection device is a stereo camera including the two image pickup units 2a and 2b as described above. The two image pickup units 2a and 2b are not parallel and are completely parallel and the same. In this case, the distance D from the focal positions of the imaging units 2a and 2b shown in FIG. 5 to the object P can be expressed by the following equation (4). In equation (4), B is the distance between the imaging means 2a and 2b (the distance between the optical axes of the optical systems 22a and 22b), f is the focal length, and C is the parallax.
D = Bf / C (4)

  However, in reality, the two image pickup means 2a and 2b are completely parallel and rarely the same, and are slightly deviated from parallel, or the image pickup means 2a and the image pickup means 2b are different in size. There are errors such as slightly different. For this reason, in order to accurately detect the position of the object, it is necessary to perform a shift correction process for correcting a shift in the rotational direction and the horizontal direction between the imaging unit 2a and the imaging unit 2b. In the present embodiment, the image processing unit 3 functions as a shift correction unit that performs a shift correction process of the imaging units 2a and 2b.

Further, in the plane coordinates (camera coordinates) on the imaging plane of the actual imaging devices 2a and 2b, the unit lengths of the horizontal axis and the vertical axis of the pixels are different, or the angle formed by the vertical axis and the horizontal axis is a right angle. There may not be. For this reason, it is necessary to perform a normalization conversion process in which the camera coordinates are normalized to obtain normalized coordinates in advance. In the present embodiment, the image processing unit 3 functions as a normalization unit that performs normalization conversion processing for converting camera coordinates into normalized coordinates using a conversion matrix described later.
In this embodiment, the deviation correction process is performed after the normalization conversion process, and the intersection transformation model is applied in the deviation correction process.

  In the present embodiment, the position detection for detecting the three-dimensional coordinates of the object (the three-dimensional position of the object, M (X, Y, Z) in FIG. 4) is specifically performed by the following method. Done.

In FIG. 4, M (X, Y, Z) is the three-dimensional coordinates of the object P. For example, the three-dimensional position of the object P when the center of the optical system is viewed as the origin is represented by XYZ coordinates. It is. On the other hand, m (x, y) represents the imaging position of the subject image of the object P imaged on the imaging surface 25 of the imaging devices 2a and 2b by two-dimensional coordinates. Here, the normalized coordinates obtained by transforming and normalizing the camera coordinates, which are the imaging positions on the actual imaging plane 25, using a transformation matrix described later are m (x, y).
In the following equations, the matrix is written in italicized capital letters (for example, P in equation (5), equation (6), etc.).

In the present embodiment, there are two imaging devices 2a and 2b. Of the imaging devices 2a and 2b, the imaging device 2a is used as a reference and imaging is performed by considering the coordinates of the imaging device 2b viewed from the imaging device 2a. Two coordinate systems, the coordinate system of the device 2a and the coordinate system of the imaging device 2b, are integrated into one to perform processing.
In each of the following formulas, the reference imaging device 2a is not given a comma, and each value relating to the other imaging device 2b is given a comma (for example, s (comma), A (comma), P _n (comma), D (comma), etc.) to distinguish the two imaging devices 2a and 2b. In the present embodiment, the case where the imaging device 2a is used as a reference will be described below as an example, but which of the plurality of imaging devices 2a and 2b is used as a reference is not particularly limited.

The relationship between the three-dimensional coordinates M (X, Y, Z) of the object P and the two-dimensional coordinates m (x, y) of the subject image can be associated via a transformation matrix, 5) and equation (6). In Expression (6), s (comma), m (comma), and M (comma) indicate that this is a value for the imaging device 2b. Moreover, in Formula (5) and Formula (6), s and s (comma) are constants, and P represents a transformation matrix (the same applies to the following formulas). In Expressions (5) and (6), M with a tilde indicates the three-dimensional coordinates of the object P in homogeneous coordinates, and m with a tilde indicates the image of the subject image. The position is shown in homogeneous coordinates (same in the following equations).

When this transformation matrix P is decomposed into a matrix A, a matrix P _n , and a matrix D, it can be expressed as the following equation (7).
Here, the matrix A is an internal matrix constituted only by variables inside the imaging means 2a and 2b, and normalized coordinates m () obtained by normalizing the camera coordinates that are the imaging positions on the actual imaging plane 25. It is a transformation matrix for performing a normalization transformation process for transformation into x, y).
The matrix P _n is a transformation matrix (projection matrix) that performs transformation between the three-dimensional coordinates M (X, Y, Z) of the object P and the two-dimensional coordinates m (x, y) of the normalized subject image. The matrix D is a matrix that represents rotation and movement (translation).

Normalized coordinates are two-dimensional coordinates obtained by a normalized camera having a focal length of 1, a pixel size of 1, and a pixel coordinate of a right angle. A projection matrix P _n representing the relationship between the three-dimensional coordinates M (X, Y, Z) of the object P and the two-dimensional coordinates m (x, y) of the subject image in the normalized camera is expressed by Expression (8). Can do. This is obtained by multiplying both sides of the previous equation (7) by the inverse matrix of the conversion matrix A. When the equation (8) is modified, the equation (9) is obtained.

Further, when the formula (5) and the formula (6) are modified by the formula (9), the formulas (10) and (11) are obtained.

Here, in the case of the conventional pinhole model, the projection matrix P _n can be expressed by Expression (12).

However, in the present embodiment, as described above, since the intersection displacement model having the coefficient k ₁ r is applied, the projection matrix P _n includes the coefficient k ₁ r and the expressions (13) and (14) Can be expressed as: Here, r is the distance from the coordinate center (x ₀ , y ₀ ) shown in FIG. 4 to m (x, y). Since Expression (14) is for the imaging apparatus 2b, a comma is added to the projection matrix P _n and the coefficient k ₁ r to distinguish it from Expression (13) regarding the imaging apparatus 2a.

In the present embodiment, as described above, since the image pickup apparatus 2a is used as a reference, the reference image pickup apparatus 2a is not rotated and moved. For this reason, the matrix D representing the rotation and movement of the imaging device 2a is expressed by Equation (15).

On the other hand, a matrix D (comma) representing the rotation and movement of the imaging device 2b is expressed by Expression (16). In equation (16), italic font r represents each component of rotation, and italic font t represents each component of translation.

Using these formulas, the formula (17) including the matrix B and the vector b can be derived by transforming the above formulas (5) and (6). Note that the matrix B and the vector b are associated with the two-dimensional coordinates m (x, y) of the subject image, as shown in equations (18) and (19) described later. Expression (17) allows m (x, y) and M (X, Y, Z) to be uniquely associated.

Here, the matrix B in the equation (17) can be expressed by the following equation (18), and the vector b can be expressed by the following equation (19).
Here, “k ₁ ” in the equation (19) is a coefficient as the function f (r) in the intersection displacement model as in the equations (13) and (14). This coefficient is determined through calibration processing by the image processing unit 3.

Here, the italic type pxx is each element of the matrix P _n D, and the italic type pxx (comma) is each element of the matrix P _n D (comma). The three-dimensional coordinates M (X, Y, Z) of the object P can be expressed by the following equation (20). For the matrix B and the vector b in the equation (20), the equations (18) and (19) The three-dimensional coordinates M (X, Y, Z) of the object P can be obtained by developing using.

In Equation (20), the matrix B ⁺ is a so-called pseudo inverse matrix represented by (B ^T B) ⁻¹ B ^T , and even when the inverse matrix of B cannot be obtained, An approximate solution can be obtained.

  Hereinafter, the position detection method in the present embodiment will be described with reference to FIGS.

As shown in FIG. 6, in the present embodiment, first, calibration processing is performed by the image processing unit 3 (step S1). In the present embodiment, the coefficient “k ₁ ” is calculated and determined by the image processing unit 3 in the course of the calibration process.

  Here, the calibration process in the present embodiment will be described with reference to FIGS.

First, the image processing unit 3 acquires a plurality of three-dimensional coordinates (X, Y, Z) of the calibration object (step S11). Next, the image processing unit 3 applies the intersection displacement model to calculate ideal values (ideal coordinates: x ₁ , y ₁ ) corresponding to the coordinates from the three-dimensional coordinates of the target object. Specifically, in the above x ₁ = ((f + f (r)) / (Zf (r))) X, y ₁ = ((f + f (r)) / (Zf (r))) Y , the coefficient _{k 1, x 1 = ((} f + k 1 r) / (Z-k 1 r)) X, y 1 = ((f + k 1 r) / (Z-k 1 r)) determining the Y (step S12 ). Since the coefficient k ₁ is not determined at this stage, an ideal value (ideal value) is provisionally determined by entering an appropriate value as the coefficient k ₁ .

Next, the image processing unit 3 applies a distortion aberration model and includes hypothetical values corresponding to the ideal values provisionally determined in step S12 (including distortion aberration related to the imaging position of the object on the imaging surface 25). (Assumed coordinates) are obtained (step S13). Specifically, a function having a certain coefficient that changes in accordance with the distance from the center, such as αr ² + βr ⁴ + γr ⁶ , is applied to the ideal coordinates. Since the coefficients α, β, and γ are not determined at this stage, appropriate values are input as the coefficients α, β, and γ, and corresponding hypothetical values are obtained (temporarily determined).

When each hypothetical value is obtained, the image processing unit 3 compares this hypothetical value with the actual measurement values (actual coordinates: x ₂ , y ₂ ) (step S14), and assumes that all the points are the actual measurement values. It is determined whether or not the value is within a predetermined error range (step S15). Specifically, the image processing unit 3 calculates a difference between the actual measurement value and the assumed value, determines whether the calculated difference exceeds a threshold value stored in the storage unit, and exceeds the threshold value. If it is determined that the measured value and the assumed value are not within the predetermined error range. The actually measured values (actual coordinates: x ₂ , y ₂ ) can be obtained from the x and y coordinates of the image that is actually imaged on the imaging surface 25 of the image sensor 21.

When it is determined that the actual measurement value and the assumed value are not within the predetermined error range (step S15; NO), the image processing unit 3 uses the coefficient k ₁ in the intersection displacement model, the coefficient α in the distortion aberration model, β and γ are changed (step S16), and the processing from step 12 to step 15 is performed again for each model whose coefficient has been changed, and the same is repeated until the actually measured value and the assumed value fall within a predetermined error range.
In such an iterative operation, the coefficient k ₁ and the coefficients α, β, and γ can be handled in the same row to some extent, so that the four coefficients α, β, γ, and the coefficient k ₁ are optimized. It is possible to search for the optimum coefficient by repeating the calculation while appropriately changing. As a method for performing such calculation, for example, DLT (Direct Liner Transform Method) or the like can be used. In the present embodiment, a process for searching for an optimum coefficient by repeating the calculation is referred to as a coefficient calculation process, and a process for this is referred to as a coefficient calculation process.

On the other hand, when it is determined that the actually measured value and the assumed value are within the predetermined error range (step S15; YES), the image processing unit 3 corrects the coefficient of the optimum function searched by the above repetitive calculation. Is stored in the storage unit (step S17). Note that storing the correction result in the storage unit is not limited to the coefficient of the function. For example, in addition to the coefficient of the function, the correspondence between the actually measured value (actual coordinates: x ₂ , y ₂ ) and the ideal value (ideal coordinates: x ₁ , y ₁ ) corresponding to the assumed value is corrected. As a result, it may be stored in the storage unit.

When correcting the image, as shown in FIG. 8, the object P is imaged by the imaging device ₂ and measured values (actual coordinates: x ₂ , y _{2) of the} object P projected onto the image sensor 21. ) Is acquired (step S21). Then, the image processing unit 3 reads out and applies the correction result obtained in steps S11 to S16 in FIG. 7 and stored in the storage unit. Thereby, ideal values (ideal coordinates: x ₁ , y ₁ ) corresponding to the actual measurement values (real coordinates: x ₂ , y ₂ ) are calculated. Based on the calculation result, the image processing unit 3 causes each measured value (actual coordinates: x ₂ , y ₂ ) of the object P to approximate an ideal value (ideal coordinates: x ₁ , y ₁ ) corresponding thereto. Then, the image is corrected (step S22), and the series of calibration processing is completed.

In the case of calibration processing using a conventional pinhole model, for example, as shown in FIG. 9, when a plurality of three-dimensional coordinates (X, Y, Z) of a calibration object are acquired (step S31), applying a pinhole model on the premise that the light beam from the object P always passes through one point of the optical system 22, and ideal values (ideal coordinates: corresponding to each coordinate) from the three-dimensional coordinates of the object. x ₁ , y ₁ ) is obtained (step S32). The ideal values (ideal coordinates: x ₁ , y ₁ ) are expressed by the above x ₁ = f (X / Z) and y ₁ = f (Y / Z).
Next, by applying a distortion aberration model, each hypothetical value corresponding to each ideal value obtained in step S32 (a hypothetical coordinate including distortion aberration relating to the imaging position of the object P on the imaging surface 25). ) Is obtained (step S33). A specific method for obtaining the assumed value is the same as that in the present embodiment.
When the assumed value is obtained, the assumed value is compared with the actually measured value (actual coordinates: x ₂ , y ₂ ) (step S34), and the actually measured value and the assumed value for all points are within a predetermined error range. It is determined whether it is within (step S35). As a result of the determination, when the actually measured value and the assumed value are not within the predetermined error range (step S35; NO), the coefficients α, β, γ in the distortion aberration model are changed (step S36), and the coefficients are changed. The processing from step 33 to step 35 is performed again with the distortion aberration model, and the same is repeated until the actually measured value and the assumed value fall within a predetermined error range.

On the other hand, when it is determined that the actually measured value and the assumed value are within the predetermined error range (step S35; YES), the image processing unit 3 corrects the coefficient of the optimum function searched by the above repetitive calculation. Is stored in the storage unit (step S37). Note that storing the correction result in the storage unit is not limited to the coefficient of the function. For example, the correspondence between the actually measured value (real coordinates: x ₂ , y ₂ ) and the ideal value (ideal coordinates: x ₁ , y ₁ ) corresponding to the assumed value or the assumed value is stored in the storage unit as a correction result. May be.

  Note that the processing for correcting an image is the same as that described with reference to FIG.

As described above, in the case of the calibration process using the conventional pinhole model, it is assumed that the light beam from the object always passes through one point of the optical system 22, and therefore, regardless of the position of the object or the like. The ideal value is fixed, and only the coefficients α, β, and γ in the distortion aberration model are changed and the calibration process is performed.
On the other hand, in the present embodiment, it is assumed that the ideal value also changes based on the image-related position on the imaging surface 25 of the object P or the position-related information regarding the position of the object with respect to the optical system 22. the coefficient k ₁ at the intersection displacement model for calculating the ideal value, coefficients in the distortion model to calculate the assumed value alpha, beta, the calibration process while changing each and γ is performed.

Next, returning to FIG. 6, the image processing unit 3 normalizes the camera coordinates of the imaging devices 2 a and 2 b by applying the internal matrix (transformation matrix) A in the equation (7) and the like, thereby obtaining the normalized coordinates. The obtained normalization conversion process is performed (step S2: normalization step). Thereafter, the image processing unit 3 applies the projection matrix P _n in Equation (7) and the like, and the matrix D representing rotation and movement (translation), thereby performing a shift correction process of the imaging units 2a and 2b (Step S3: Deviation correction step). In this deviation correcting process, the image processing unit 3 applies the intersection displacement model, performing a calculation incorporating the coefficients k ₁ calculated in the calibration process the projection matrix P _n.

  Then, after performing the normalization conversion process and the deviation correction process in this way, the image processing unit 3 detects the position (three-dimensional coordinates M (X, Y, Z)) of the object P that is the position detection target. (Step S4). In the present embodiment, the position detection process includes step S2 (normalization process) and step S3 (deviation correction process).

  As described above, according to the present embodiment, when detecting the position of the object P, the intersection of the light beam incident on the optical system 22 and the optical axis of the optical system 22 moves back and forth in the extending direction of the optical axis. Since the intersection displacement model is applied, even when the distance from the object P to the imaging devices 2a and 2b is not known or when a plurality of objects P having different distances are mixed in one screen, one model is used. The position of the object P can be detected easily and with high accuracy.

Further, a light beam incident on the optical system 22 based on an actual measurement value regarding the imaging position of the object P on the imaging surface 25 obtained through the optical system 22 or position-related information regarding the position of the object P with respect to the optical system 22. An ideal value (ideal coordinates: x ₁ , y ₁ ) is calculated by applying an intersection displacement model in which the intersection with the optical axis of the optical system 22 is back and forth in the extending direction of the optical axis. For this reason, even if the intersection of the light beam incident on the optical system 22 and the optical axis of the optical system 22 changes according to the position of the object P, the correct ideal value (ideal coordinates: x ₁ , y ₁ ) can be determined.
Then, an assumed value (a value including distortion) corresponding to the ideal value calculated in this way is calculated, and this is compared with an actual measurement value, so that the distance from the object P to the optical system 22 is known. Even when there is no object P or when there are a plurality of objects P having different distances on one screen, it is possible to easily cope with changes in distortion without preparing a correction curve for different distortion aberrations for each distance. Therefore, it is possible to correct distortion aberration with high accuracy regardless of the distance of the object P.

In the present embodiment, the ideal values (ideal coordinates: x ₁ , y ₁ ) are calculated, and the assumption value is first obtained by applying the function of the distortion aberration model. However, the processing procedure is as follows. It is not limited to what was illustrated here. For example, a graph representing the distortion aberration corresponding to the image height of the object P as shown in FIG. 10 by taking a plurality of actually measured values (actual coordinates: x ₂ , y ₂ ) and calculating the distortion aberration corresponding thereto. (For example, a solid line in FIG. 10) is created, and this is replaced with a function such as αr ² + βr ⁴ + γr ⁶ that changes according to the distance from the center (for example, a dotted line in FIG. 10). A method may be used in which coefficients such as α, β, and γ are repeatedly calculated to select a coefficient that best matches the actual distortion aberration.

Further, in FIG. 10 above, there is a case where a distortion aberration that shifts closer to the center (x ₀ , y ₀ ) of the image than the ideal value appears (an example in which the distortion aberration value is a negative value). An example is shown in which the aberration gradually increases from the center (x ₀ , y ₀ ) of the image to the outside (the direction in which the image height increases) (an example in which the value increases as it goes to the outside). The appearance of distortion varies depending on the configuration of the optical system and the like, and does not always change as illustrated here.
Depending on the optical system, the distortion gradually increases from the center of the image to the outside, but when the distortion reaches a certain point, the distortion becomes small, or the distortion increases from the center of the image to the outside. There may be a case where the change gradually decreases. In addition, when distortion aberration that shifts away from the center (x ₀ , y ₀ ) from the ideal value as it goes outward from the center (x ₀ , y ₀ ) of the image appears (the value of distortion aberration is positive) There is also an example in which the value of In any case, the intersection displacement model described in this embodiment can be applied, and appropriate calibration processing and position detection processing can be performed.

Further, in the present embodiment, the function f (r) in the intersection displacement model is a function that changes only linearly according to the image height r (imaging position x, y) (“k ₁ r”). Although an example is shown, the function in the intersection displacement model is not limited to this.

For example, the following function may be used as the function f (r) in the intersection displacement model.
x = ((f + (k ₁ r + k ₁ r ² +...)) / (Z− (k ₁ r + k ₁ r ² +...))) X, y = ((f + (k ₁ r + k ₁ r ² +...)) / (Z− (k ₁ r + k ₁ r ² +...))) Y
When such a function is used, for example, as r increases, the value gradually increases partway through, but when the value decreases from a certain point or when the value increases, little by little. It is possible to deal with various models such as a case where it does not increase but increases rapidly in proportion to the square of r.

For example, the following functions may be used.
x = ((f + k ₁ θ) / (Z−k ₁ θ)) X, y = ((f + k ₁ θ) / (Z−k ₁ θ)) Y
This expresses the equation not by the x and y coordinates such as the image height r but by the angle θ incident on the optical system 22 from the object P. Since the x and y coordinates and the angle θ have a corresponding relationship with each other, the x and y coordinates can be determined once θ is determined.
In general, an ideal value (ideal imaging position) with respect to the angle of the light beam incident on the optical system 22 from the object P can be expressed by f tan θ as shown in FIG. However, for example, when photographing is performed using a wide-angle lens of about 180 degrees as the optical system 22, the optical system 22 is at an angle that the light flux from the object P is almost 180 degrees as shown in FIG. Since the image forming position is considerably distant by being incident on, the image forming position is generally represented by fθ. In such a case, since it is not realistic as a model if it is shown at the imaging position, it is effective to show it at the angle θ instead of the imaging position. In terms of the angle θ, if the angle on the incident side is 180 degrees, the angle on the exit side is also 180 degrees, which can be expressed easily. For this reason, especially when a wide-angle lens is used, the above-mentioned function model that changes linearly according to the angle θ is suitably adapted.

Furthermore, for example, for the case where the position of the object P is known as described below, a function model that changes in accordance with the position can be used.
x = ((f + k ₁ (R / Z)) / (Z−k ₁ (R / Z))) X, y = ((f + k ₁ (R / Z)) / (Z−k ₁ (R / Z)) )) Y
The function model applied to the intersection displacement model is not limited to the one shown here, and can be modified as appropriate.

  In the present embodiment, the case where the normalization conversion process, the shift correction process, and the coefficient calculation process in the position detection method are all performed by the image processing unit 3 has been described as an example. It is not limited to the case where it performs, You may make it perform each process in a some function part. In addition, the coefficient applied in the position detection process is not limited to the case where the coefficient is calculated in the position detection device. For example, the image processing unit of the position detection device uses the coefficient calculated in the external device. Detection processing may be performed.

  Further, in the present embodiment, the case where the image of the object for calibration at the time of performing the calibration process is acquired by the imaging devices 2a and 2b has been described as an example. However, the image data to be calibrated is externally acquired. It is good also as input possible from. In addition, the position detection device may be configured separately from the imaging device. In this case, the position detection device performs position detection processing for detecting the position of the object based on image data acquired from the outside.

  In addition, it is needless to say that the present invention is not limited to the above embodiment and can be modified as appropriate.

[Second Embodiment]
Next, a second embodiment of the position detection device and the position detection method according to the present invention will be described with reference to FIGS. The position detection device in the second embodiment includes an image processing unit (not shown) similar to that in the first embodiment, and the position detection method is executed by the image processing unit. Since only the procedure of the calibration process for acquiring the coefficient k ₁ is different from that of the first embodiment, the following description will particularly focus on differences from the first embodiment. The apparatus configuration is the same as that of the first embodiment, and a description thereof will be omitted.

In the position detection method in this embodiment, as in the first embodiment, firstly, it calculates a coefficient k ₁ by performing the calibration process. The calibration process in the present embodiment will be described with reference to FIG.

As shown in FIG. 9, distortion (measured value) corresponding to the distance of the object is acquired (measured) in advance for each distance of the object (step S41).
FIG. 9 is a graph plotting distortion aberrations (actual measurement values) for each acquired distance. Then, an arbitrary distance (for example, when the distance of the object P is 0.3 m) is selected as the representative distance (step S42). For the distortion aberration (actually measured value) at this representative distance, a coefficient (for example, αr ² + βr ⁴ + γr ⁶ ) of a distortion aberration model (for example, αr ² + βr ⁴ + γr ⁶ ) is obtained by the same method as that shown in the first embodiment. Determines the coefficients α, β, γ) (step S43).

Next, an intersection displacement model similar to that of the first embodiment is applied to the distortion at the representative distance, and the intersection of the light beam incident on the optical system 22 and the optical axis of the optical system 22 is the extension of the optical axis. It is calculated what the distortion aberration (assumed value) for each distance becomes when the back-and-forth movement is taken into consideration (step S44). Note that by calculating the difference in the distance of the object P, the distance of the object P is 0.5 m from the distortion (assumed value) in the case of the representative distance (for example, the distance of the object P is 0.3 m). It is possible to calculate the distortion aberration (assumed value) in the case of 0.7, the distortion aberration (assumed value) in the case of 0.7 m, the distortion aberration (assumed value) in the case of 0.9 m, respectively.
At this stage, since the coefficient of the intersection displacement model (for example, the coefficient k _{1 in} the first embodiment) cannot be determined, an appropriate value is temporarily determined as the coefficient k ₁ and the calculation is performed.

  Then, the distortion aberration (measured value) for each distance acquired in step S41 is compared with the distortion aberration (assumed value) for each distance obtained by applying the intersection displacement model (step S45), and the measured value and the assumed value are compared. Are within a predetermined error range (step S46). Specifically, the difference between the actually measured value and the assumed value is calculated, and it is determined whether or not the calculated difference exceeds the threshold value stored in the storage unit. And the assumed value are not within the predetermined error range.

The case where the measured value and the assumed value is determined not within the predetermined error range; (step S46 NO), change the coefficient k ₁ at the intersection displacement model (step S47), coefficients of the modified model Then, the processing from step 43 to step 46 is performed again, and the same is repeated until the actually measured value and the assumed value are within the predetermined error range. In the present embodiment, a process for searching for an optimum coefficient by repeating the calculation is referred to as a coefficient calculation process, and a process for the process is referred to as a coefficient calculation process.

  On the other hand, when it is determined that the actually measured value and the assumed value are within the predetermined error range (step S46; YES), the image processing unit uses the coefficient of the optimum function searched by the above repetitive calculation as the correction result. Store in the storage unit (step S48). Note that storing the correction result in the storage unit is not limited to the coefficient of the function. For example, the correspondence between the actually measured value (measured distortion aberration) and the assumed value (calculated distortion aberration) may be stored in the storage unit as a correction result.

  Note that the processing for correcting an image is the same as that described in FIG. 8 of the first embodiment, and thus the description thereof is omitted.

Then, the image processing unit, using the coefficient k ₁ calculated in the calibration process, performs the position detecting process for detecting the position of the object. Specifically, as in the first embodiment, first, normalization conversion processing is performed to obtain normalized coordinates by normalizing the camera coordinates of the imaging apparatus (normalization step). Thereafter, the image processing unit performs error correcting process of the image pickup means by applying an intersection displacement model including the coefficients k ₁ (deviation correcting step), to detect the position of the object.

As described above, according to the present embodiment, the assumed value for each distance is acquired by applying the intersection displacement model, so that the distance from the object to the imaging device is not known or in one screen Even when a plurality of objects having different distances are mixed, highly accurate distortion correction can be easily performed.
Also performs deviation correction processing between the image pickup means using the coefficient k ₁ of the intersection displacement model obtained in the course of the calibration process, and detects the position of the object, when applying the conventional pinhole model and Compared to this, it is possible to perform position detection that is more flexible and accurate.

  It is to be noted that the present invention is not limited to the above-described embodiment, but can be changed as appropriate as in the first embodiment.

It is a block diagram which shows schematic structure of the position detection apparatus used by this embodiment. It is a figure explaining the ideal value and measured value in the conventional pinhole model. It is a figure explaining the ideal value and measured value in an intersection displacement model. It is a figure explaining the relationship between the three-dimensional position of a target object, and the position on an image plane. It is a figure explaining the distance from a focus position in case there exist two imaging devices to a target object. It is a flowchart which shows the position detection process in 1st Embodiment. It is a flowchart which shows the calibration process shown in FIG. It is a flowchart which shows the process at the time of correct | amending an image. It is a flowchart which shows the calibration process in the conventional pinhole model. It is a graph explaining substitution to the function of distortion aberration. FIG. 11A is a diagram for explaining the relationship between the incident angle of a light beam and an ideal value in the case of a general photographing lens. FIG. 11B is a diagram for explaining the relationship between the incident angle of the light beam and the ideal value in the case of the wide-angle lens. It is a flowchart which shows the calibration process in 2nd Embodiment. It is a graph explaining substitution to the function of distortion aberration.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Position detection apparatus 2a, 2b Image pick-up device 21a, 21b Image pick-up element 22a, 22b Optical system 3 Image processing part 25 Imaging surface P Object

Claims (6)

  The optical system based on actual measurement values on the imaging position of an object on the imaging plane obtained by a plurality of imaging means provided with an optical system and imaging the object or position related information on the position of the object with respect to the optical system Position detection for detecting the position of the object imaged by the imaging means by applying an intersection displacement model in which the intersection of the light beam incident on the optical system and the optical axis of the optical system is back and forth in the extending direction of the optical axis A position detecting device comprising means. The position detecting unit includes a normalizing unit that performs normalization conversion processing to obtain normalized coordinates by normalizing camera coordinates, and a shift that performs shift correction processing of the plurality of imaging units after being normalized by the normalizing unit. Correction means,
The position detection apparatus according to claim 1, wherein the intersection displacement model is applied in the shift correction processing in the shift correction unit.   The position detection device according to claim 1, further comprising coefficient calculation means for calculating a coefficient in the intersection displacement model.   The optical system based on actual measurement values on the imaging position of an object on the imaging plane obtained by a plurality of imaging means provided with an optical system and imaging the object or position related information on the position of the object with respect to the optical system Position detection for detecting the position of the object imaged by the imaging means by applying an intersection displacement model in which the intersection of the light beam incident on the optical system and the optical axis of the optical system is back and forth in the extending direction of the optical axis A position detection method comprising a step.   The position detection step includes a normalization step of normalizing a camera coordinate to obtain a normalized coordinate, and a shift for performing a shift correction process of the plurality of imaging units after being normalized by the normalization step The position detection method according to claim 4, further comprising: a correction step, wherein the intersection displacement model is applied in the shift correction step.   The position detection method according to claim 4, further comprising a coefficient calculation step of calculating a coefficient in the intersection displacement model.

Images