JP2010122976A - Image correction evaluation device and image correction evaluation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve robustness or accuracy of evaluation by quantifying success/failure of correction based on image information. <P>SOLUTION: An image correction evaluation device for evaluating the success/failure of the image correction in an imaging parameter of an imaging means from an image obtained by the imaging means has: an image input means for fetching the image of a correction target photographed by the imaging means; a pattern storage means for holding a figure characteristic of a three-dimensional pattern including a curve or a straight line that is a figure for correction; a projection means for obtaining the figure characteristic projected onto an image coordinate system by use of the figure characteristic and the input imaging parameter; a scan line segment setting means for setting one or more points on a projection image of the figure for correction based on a figure projected by the projection means, and setting a line segment passing through each set point; and an evaluation value arithmetic means for acquiring a pixel value of each pixel on the line segment in the image fetched by the image input means, and calculating an evaluation value quantified with the success/failure of the correction by use of the obtained pixel value. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image calibration evaluation apparatus and an image calibration evaluation program, and more particularly to an image calibration evaluation apparatus and an image calibration for improving the robustness and accuracy of evaluation by quantifying the success or failure of calibration based on image information. Regarding the evaluation program.

  In conventional camera calibration, a predetermined or previously measured pattern is photographed by a camera to be calibrated, and camera parameters are obtained by an analytical method or a method of repeatedly performing a convergence calculation from the shape of the pattern image.

  Here, as a representative example of the method based on the repeated convergence calculation described above, for example, the degree of coincidence between the actually observed pattern image and the observed pattern image estimated from the camera parameters and the pattern model is evaluated. However, there is a method of sequentially correcting the camera parameters so that the degree of coincidence is maximized. In addition, as the above-described pattern, a method using a correspondence relationship between a plurality of observation points and points on the model (see, for example, Non-Patent Document 1), a plurality of observed lines (straight lines, line segments, curves, etc.) ) And a line on the model.

  Further, in the method using the plurality of point information described above, for example, a representative point (centroid, etc.) of a specific pattern such as a point where lines intersect, a vertex of a broken line, a circle or a rectangle is extracted as a feature point, and correspondence with a model The mainstream is to seek relationships.

On the other hand, in the method using a plurality of line information, for example, parameters of a straight line or a curve are calculated by Hough transform or generalized Hough transform, and correspondence between the parameters and the model is performed (for example, Patent Documents). 1).
Roger Y. Tsai: “A Versatile Camera Calibration Technology for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shel TV Camera and Erenso”. RA-3, no. 4, pp. 323-344, August 1987. JP 2004-234333 A

  By the way, in the above-described conventional camera calibration method, feature points, straight lines, and curves in an image are extracted, and the coordinates and parameters are obtained as an initial stage, and then the points and lines obtained by projecting the model are converted. The distance between the coordinates and parameters and the coordinates and parameters obtained from the observed image is compared.

  Here, the point and line extraction processing is usually a non-linear operation on the image and tends to lose much of the original image information. For this reason, if the extraction of points and lines from the image becomes unstable, the success or failure of the calibration cannot be evaluated, and the camera parameters cannot be estimated.

  The present invention has been made in view of the above-described problems, and an image calibration evaluation apparatus and an image calibration evaluation program for improving evaluation robustness and accuracy by quantifying the success or failure of calibration based on image information. The purpose is to provide.

  Specifically, in the camera calibration evaluation in the present invention, in order to avoid the loss of information in the point and line extraction processing, the calibration is performed by directly using the pixel value information of the image captured by the camera to be calibrated. The purpose is to be able to determine the success or failure of.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

  According to the first aspect of the present invention, in an image calibration evaluation apparatus that evaluates the success or failure of image calibration in the imaging parameters of the imaging unit from an image obtained by the imaging unit, an image to be calibrated captured by the imaging unit is obtained. An image input unit to be captured, a pattern storage unit that holds a graphic feature of a three-dimensional pattern including a straight line or a curve that is a calibration graphic, and the graphic feature and the input imaging parameter are used to project the image onto the image coordinate system. A scanning means for setting one or more points on the projected image of the calibration graphic based on the graphic projected by the projection means for obtaining the graphic feature, and a line passing through each set point based on the graphic projected by the projection means In the image captured by the line segment setting means and the image input means, the pixel value of each pixel on the line segment is acquired, and the success or failure of the calibration is quantified using the acquired pixel value. And having an evaluation value calculating means for calculating an evaluation value.

  According to the first aspect of the present invention, the robustness and accuracy of the evaluation can be improved by quantifying the success or failure of the calibration based on the image information.

  According to a second aspect of the present invention, the evaluation value calculating means is preset in accordance with the position on each line segment to the pixel value of each pixel on all line segments obtained by the scan line segment setting means. An integral value or a sum is obtained by multiplying the weighting factor.

  According to the second aspect of the present invention, for example, a higher-precision evaluation value can be calculated by increasing the weight of the short distance portion necessary for image calibration.

  According to a third aspect of the present invention, the projection unit acquires geometric feature information corresponding to the imaging parameter as the graphic feature from a plurality of pieces of geometric feature information stored in the pattern storage unit, and acquires the acquired geometric feature. The feature information is projected and converted to output geometric feature information in an image coordinate system.

  According to the third aspect of the present invention, it is possible to easily calculate a more accurate evaluation value by using the geometric feature information.

  In the invention described in claim 4, the graphic feature includes one or more representative points on a pattern including a straight line or a curve obtained by the imaging unit, and a tangential direction of the straight line or the curve at the representative point. It is characterized by that.

  According to the fourth aspect of the present invention, it is possible to easily perform image calibration by utilizing the graphic features including the representative point of the pattern and the tangent direction at the representative point.

  In the invention described in claim 5, the scan line segment setting means sets, as a scan line segment, a line segment that passes through the image for each representative point in the graphic feature and is orthogonal to the tangential image at the representative point. It is characterized by doing.

  According to the fifth aspect of the present invention, it is possible to easily acquire the scan line segment necessary for the calibration evaluation easily.

  According to a sixth aspect of the present invention, there is provided an image calibration evaluation program for evaluating success or failure of image calibration in an imaging parameter of the imaging unit from an image obtained by the imaging unit. An image input unit that captures an image of the image, a pattern storage unit that holds a graphic feature of a three-dimensional pattern including a straight line or a curve that is a calibration graphic, and is projected onto an image coordinate system using the graphic feature and the input imaging parameter A scanning means for setting one or more points on the projected image of the calibration graphic based on the graphic projected by the projection means for obtaining the graphic feature, and setting a line segment passing through each set point In the image captured by the line segment setting unit and the image input unit, the pixel value of each pixel on the line segment is acquired, and the acquired pixel value is used. To function as an evaluation value calculation means for calculating a quantified evaluation value result of calibration Te.

  According to the invention described in claim 6, the robustness and accuracy of the evaluation can be improved by quantifying the success or failure of the calibration based on the image information. Furthermore, image calibration evaluation can be easily realized by installing an execution program in a computer.

  According to the present invention, the robustness and accuracy of evaluation can be improved by quantifying the success or failure of calibration based on image information.

<Outline of the present invention>
The present invention relates to a calibration evaluation apparatus for evaluating whether or not a camera parameter (imaging parameter) consisting of one or more of information on the position, orientation, or focal length of a camera as an imaging means is correct, and in particular, a calibration target. The present invention relates to an apparatus that performs evaluation based on an image captured by a camera.

  Specifically, the present invention is obtained by a camera that has been photographed when calibrating the installation position, posture, or focal length of a camera that is an imaging means using a known straight line or curve pattern included in the photographing space. The degree of success or failure of calibration is quantified based on image information.

  For example, a pattern including a straight line or a curve is photographed by a camera, and then one or more points (representative points) are obtained on the curve group based on a known model of the straight line or curve pattern (hereinafter, curve group). Further, the representative point and the tangential direction of the curve at the representative point are projected onto the image coordinate system based on the camera parameter (camera installation position, posture or focal length) to be evaluated.

  Next, regarding the projection image of each representative point, a line segment (scan line segment) that passes through the above-described projection image and is orthogonal to the tangential projection image is obtained. Also, in the captured image, all the pixels on the scan line segment are obtained by multiplying the pixel value by a weighting factor set in advance according to the position on each scan line segment, and obtaining the integral value or sum, thereby calibrating. An evaluation value obtained by quantifying the success or failure of is calculated and output.

  In the following, a preferred embodiment of an image calibration evaluation apparatus and an image calibration evaluation program according to the present invention having the above-described features will be described in detail with reference to the drawings.

<Image calibration evaluation device: functional configuration>
FIG. 1 is a diagram illustrating an example of a functional configuration of a video configuration evaluation apparatus according to the present embodiment. The image calibration evaluation apparatus 10 shown in FIG. 1 includes an image input unit 11, a pattern storage unit 12, a projection unit 13, a scan line segment setting unit 14, and an evaluation value calculation unit 15.

  The image input means 11 reads an image photographed by a camera as an imaging means that is a calibration target, and stores the read image in a built-in memory or the like. Note that the image input unit 11 stores, for example, a pixel value I (x, y) with respect to the image coordinates (x, y) in a form that can be referred to when storing the image.

  Here, x and y described above are integer values. For example, x represents a horizontal component of image coordinates, and y represents a vertical component of image coordinates. At this time, for example, the image input means 11 sets the image coordinates of the pixel at the upper left corner of the image as x = 0, y = 0, and increases the value of x by 1 every time one pixel is moved to the right. The image coordinate system can be defined so that the value of y is increased by 1 every time one pixel is moved.

  The camera to be calibrated may be, for example, a still camera or a video camera. The output image may be a digital image or an electronic analog image. Here, when the camera to be calibrated has a digital image output, the image input means 11 extracts the pixel value at each pixel position from the data output from the camera, and stores the extracted pixel value in the memory. It is assumed that the pixel value I (x, y) with respect to the image coordinates (x, y) can be referred to. When the camera to be calibrated has an electronic analog image output, the image input means 11 performs analog / digital conversion on the image output from the camera, acquires pixel values at each pixel position, and Is stored in the memory so that the pixel value I (x, y) with respect to the image coordinate (x, y) can be referred to.

  Furthermore, as a camera to be calibrated, for example, a method of recording an image by a chemical reaction (for example, a silver salt photograph) can be used. In such a case, the image input means 11 is a film, dry plate, photographic paper photographed and developed by a camera, or a film, dry plate, or photographic paper obtained by optically enlarging, reducing, or magnifying these images with a photographic scanner. Read. Further, the image input means 11 stores the read pixel value in the memory so that the pixel value I (x, y) with respect to the image coordinate (x, y) can be referred to.

  The pattern storage means 12 is a geometric feature (graphic feature) of a camera pattern (calibration pattern) used for camera calibration based on a straight line group, a curve group, or a combination thereof (hereinafter referred to as a curve group) drawn or installed in real space. Is a memory for storing.

  Here, the calibration pattern may be a pattern created on the basis of preset dimensions, angles, and curvatures, and is obtained by measuring the dimensions and shapes of artifacts and natural objects already existing in real space. May be.

  Here, FIG. 2 is a diagram illustrating an example of a camera pattern used for camera calibration. For example, a diagram whose length, arrangement, and orientation are known as shown in FIG. 2 can be used as the calibration pattern 21. Therefore, for example, a line of a court used for a sports competition such as a soccer field or a line on a stadium in a rugby field that can be obtained from a video taken from a camera can be used as the calibration pattern 21.

  In addition, when using a sport competition court line as the calibration pattern 21, the length, arrangement, posture, curvature, etc. of the curve group are measured in advance, or values determined in the competition rules are used, or they are used. It shall be due to the combined use of

  That is, the pattern storage unit 12 stores geometric feature information including a numerical group representing a geometric feature of a curve group constituting a calibration pattern. Here, the above-described geometric feature information refers to the coordinates of one or more representative points on a curve (including a straight line, a half line, and a line segment, and so on) constituting the calibration pattern, and a curve at each representative point. And information that can describe the tangent direction of.

  For example, as geometric feature information, three-dimensional coordinates of both end points of a line segment included in the calibration pattern, circle center and radius, ellipse center, minor radius, major radius and major axis phase angle, arc center, The radius, the phase angle at the start point and the phase angle at the end point, the coordinates of the representative point on the curve, the tangent direction at the representative point, and the like can be used.

  Here, FIG. 3 is a diagram showing an example of geometric feature information in the present embodiment. The geometric feature information shown in FIG. 3 shows an example of the representative point set on the curve (in this case, the line segment) of the calibration pattern 21 shown in FIG. 2 and the tangential direction at the representative point. That is, the geometric feature information can be defined by one or more representative points 31 on the line of the calibration pattern 21 and the tangential direction 32 at the representative point 31.

  The installation interval of the plurality of representative points may be an interval based on a line that exists in a preset image, or may be an interval that is based on a line that exists in an actual imaged location. The intervals may be equal or unequal. That is, the number and position of the representative points in the present invention are not particularly limited. Further, the tangential direction 32 at the representative point 31 may be on the line segment of the calibration pattern 21.

  The projection unit 13 is based on the camera parameter (imaging parameter) θ input corresponding to the camera (imaging unit) that captured the image input from the image input unit 11, and the geometric feature information stored in the pattern storage unit 12. The geometric feature information of the corresponding camera parameter is extracted from the image, the extracted geometric feature information is projected and converted, and the geometric feature information in the image coordinate system is output.

  Here, the above-described camera parameter θ includes, for example, one or more parameters of the camera installation position, posture, and focal length (or angle of view) of a lens (including a reflector and a pinhole, the same applies hereinafter). Shall be included. In addition to these, the camera parameter θ includes at least one of parameters such as a lens distortion parameter, an image coordinate of an intersection of the optical axis and the image plane, and an inclination of the optical axis with respect to the image plane. It doesn't matter.

  The projection conversion in the projection unit 13 is a conversion for performing a preset projection corresponding to the configuration and shape of the lens and the imaging surface of the camera to be calibrated. For example, in the case of projecting onto a flat image plane using a pinhole or a lens or reflector that can be approximated by a pinhole model, perspective projection is performed. In the case of using a fisheye lens, a projection model that is most suitable for the relationship between the lens to be used and the imaging plane, such as equidistant projection, equisolid angle projection, or orthographic projection, is applied.

  Here, a mathematical expression for converting the coordinates (X, Y, Z) in the real space into the image coordinates (x, y) by the projection unit 13 of the camera parameter θ is set as the following expression (1).

ここで、例えばパターン記憶手段１２には、曲線上の第ｉ番目（但し、ｉ∈｛０，１，Ｎ−１｝、Ｎは１以上の整数）の代表点の座標（Ｘ_ｉ，Ｙ_ｉ，Ｚ_ｉ）、及びその代表点における接線方向（Ｕ_ｉ，Ｖ_ｉ，Ｗ_ｉ）が記憶されている。投影手段１３は、例えば投影変換が透視投影による場合、以下に示す（２）式〜（４）式により代表点の像（ｘ_ｉ，ｙ_ｉ）、及び代表点における接線方向の像（ｕ_ｉ，ｖ_ｉ）を取得することができる。

Here, for example, the pattern storage means 12 stores the coordinates (X _i , Y _i ) of the i-th representative point on the curve (where i∈ {0, 1, N−1}, N is an integer of 1 or more). , Z _i ) and the tangential direction (U _i , V _i , W _i ) at the representative point are stored. For example, when the projection conversion is based on perspective projection, the projection unit 13 uses the following expressions (2) to (4) to represent the representative point image (x _i , y _i ) and the tangential image (u _i ) at the representative point. , V _i ) can be obtained.

スキャン線分設定手段１４は、投影手段１３により得られた校正用パターンの像に関する情報（例えば、代表点の像、及び代表点における接線方向の像）に基づき、画像平面内において１以上の線分を設定する。この線分は、後述する評価値演算手段１５における評価値の計算に用いるためのものであり、画素値の走査を行うための領域として用いられる。

The scan line segment setting unit 14 is based on information about the image of the calibration pattern obtained by the projection unit 13 (for example, the image of the representative point and the image in the tangential direction at the representative point), and one or more lines in the image plane. Set the minutes. This line segment is used for calculating an evaluation value in the evaluation value calculation means 15 described later, and is used as an area for scanning pixel values.

For example, the scan line segment setting means 14 passes through the image (x _i , y _i ) of each representative point in the above-described equation (2) and is orthogonal to the image (u _i , v _i ) in the tangential direction at the representative point. Set the line as the scan line.

Further, the scan line segment can be expressed as a locus of (p _i (t), q _i (t)) using t as a parameter, for example, as shown in the following equation (5).

なお、上述した（５）式において、Ｔ_０及びＴ_１は、Ｔ_０＜Ｔ_１なる実数の定数とし、好ましくはＴ_０＜０、Ｔ_１＞０とする。例えば、Ｔ_０＝−１０、Ｔ_１＝１０とおく。

In the above-described equation (5), T ₀ and T ₁ are real constants such that T ₀ <T ₁ , and preferably T ₀ <0 and T ₁ > 0. For example, T ₀ = −10 and T ₁ = 10.

  Here, FIG. 4 is a diagram illustrating an example of the scan line segment. As shown in FIG. 4, the scan line segment passes through the image of each representative point 31 described above with respect to the projection image 41 of the calibration graphic, and a line segment orthogonal to the tangential image is set as the scan line segment 42. .

  The installation interval of the plurality of scan line segments 42 may be, for example, an interval based on a projection image 41 in a preset image, or may be an interval based on a line that is actually captured. . The intervals may be equal or unequal. That is, the number and position of scan line segments in the present invention are not particularly limited.

  The evaluation value calculation means 15 scans all line segments set by the scan line segment setting means 14 in the image I (x, y), and calculates an evaluation value E based on the pixel value group of the scanned pixels. Output.

  For example, the evaluation value calculation means 15 can calculate the evaluation value E by the following equation (6).

ここで、上述した（６）式において、ｆ（Ｉ）は画素値Ｉを引数に取り、実数を返す関数である。

Here, in the above equation (6), f (I) is a function that takes the pixel value I as an argument and returns a real number.

For example, the pixel value I red I _R, green I _G, and when a vector consisting of the respective color components of blue I _B, a linear combination consisting shown below (7), to define f (I) Can do.

ここで、上述した（７）式において、ｋ_Ｒ、ｋ_Ｒ、及びｋ_Ｂは、何れも実数とし、ｋ_Ｒ、ｋ_Ｒ、及びｋ_Ｂの全てが０とはならないものとする。

Here, in Equation (7) described above, k _R , k _R , and k _B are all real numbers, and k _R , k _R , and k _B are not all zero.

  Further, for example, f (I) is a value to be returned as shown in the following equation (8) depending on whether the pixel value of I is included in the predetermined area C of the color space or not. It may be changed.

ここで、上述した（８）式は、ハードクロマキーによるキー値生成と等価である。したがって、例えば、校正対象の映像としてサッカー場を撮影している場合には、その映像から得られる画像の色が白色（コートのライン）か否か（例えば、緑色（サッカー場の芝生）等）を判断し、白線の場合は「１」とし、それ以外の場合（例えば、緑色等）は「０」とすることができる。

Here, the above-described equation (8) is equivalent to key value generation using a hard chroma key. Therefore, for example, when shooting a soccer field as a video to be calibrated, whether or not the color of the image obtained from the video is white (court line) or not (for example, green (soccer field grass)) In the case of a white line, “1” can be set, and in other cases (for example, green or the like), “0” can be set.

  Further, all the values returned by f (I) may be defined according to each value of I. That is, when the pixel value I is discrete, for example, f (I) can be realized using a lookup table for the pixel value I.

  Further, the weight function w (t) in the above-described equation (6) is a weight function that changes according to the parameter t representing the position of the point on the scan line segment. Therefore, it is preferable that the weight function w (t) satisfies the following expression (9).

ここで、図５は、引数が連続値である場合の重み関数の一例を示す図である。例えば、重み関数ｗ（ｔ）として図５（ａ），（ｂ）に示すような関数を用いることができる。具体的には、例えば図５（ａ）の関数では、以下に示す（１０）式により表すことができる。

Here, FIG. 5 is a diagram illustrating an example of a weighting function when the argument is a continuous value. For example, a function as shown in FIGS. 5A and 5B can be used as the weight function w (t). Specifically, for example, the function of FIG. 5A can be expressed by the following equation (10).

なお、図５（ａ）と同様に、図５（ａ）の関数を反転させた図５（ｂ）の関数を用いてもよい。

Similar to FIG. 5A, the function of FIG. 5B obtained by inverting the function of FIG. 5A may be used.

  Furthermore, FIG. 6 is a diagram illustrating an example of a weighting function when an argument different from that in FIG. 5 is a continuous value. For example, as the weighting function w (t), a function as shown in FIGS. 6A and 6B can be used using a cosine curve (cosine wave) in addition to the continuous function shown in FIG. Specifically, for example, the function of FIG. 6A can be expressed by the following equation (11).

なお、図６（ａ）と同様に、図６（ａ）の関数を反転させた図６（ｂ）の関数を用いてもよい。

As in FIG. 6A, the function of FIG. 6B obtained by inverting the function of FIG. 6A may be used.

  Further, the integral part of the above-described equation (6) may be discretized and replaced with a sum operation. In this case, the evaluation value E can be expressed by, for example, the following expression (12).

ここで、ｔ_ｊ（ｊ∈｛０，１，…，Ｊ−１｝）は、Ｔ_０≦ｔ_ｊ≦Ｔ_１なる数列である。このとき、重み関数ｗ（ｔ）は、例えば、以下に示す（１３）式で表すことが好ましい。

Here, t _j (j∈ {0, 1,..., J−1}) is a sequence of T ₀ ≦ t _j ≦ T ₁ . At this time, the weighting function w (t) is preferably expressed by, for example, the following expression (13).

ここで、上述した（１３）式において、ｔ_ｊとしては、例えばＴ_０≦τ≦Ｔ_１を満たす実数τを、ある一定の間隔で昇順又は降順に並べたものとすることができる。なお、実数τは、画像負荷等を考慮して設定することができる。

Here, in the above-described equation (13), as t _j , for example, real numbers τ satisfying T ₀ ≦ τ ≦ T ₁ may be arranged in ascending or descending order at a certain interval. The real number τ can be set in consideration of the image load and the like.

Further, for example, integers τ satisfying T ₀ ≦ τ ≦ T ₁ can be arranged in ascending order as t _j , and the following equation (14) can be obtained.

ここで、ｆｌｏｏｒ［Ｔ］は、Ｔより大きくない最大の整数（即ち、Ｔに対する床関数）を表し、ｃｅｉｌ［Ｔ］はＴより小さくない最小の整数（即ち、Ｔに対する天井関数）を表す。このとき、例えば、Ｔ_０＝−１０，Ｔ_１＝１０の場合には、以下に示す（１５）式のようになる。

Here, floor [T] represents the largest integer not greater than T (ie, the floor function for T), and ceil [T] represents the smallest integer not less than T (ie, the ceiling function for T). At this time, for example, when T ₀ = −10 and T ₁ = 10, the following equation (15) is obtained.

また、上述した（１２）式は、以下に示す（１６）式のように表すことができる。

Moreover, (12) Formula mentioned above can be represented like (16) Formula shown below.

ここで、図７は、引数が離散値である場合の重み関数の一例を示す図である。このとき、重み関数ｗ（ｔ）として、例えば図７に示す関数、即ち、以下に示す（１７）式のような関数を用いることができる。

Here, FIG. 7 is a diagram illustrating an example of the weighting function when the argument is a discrete value. At this time, as the weighting function w (t), for example, a function shown in FIG. 7, that is, a function like the following expression (17) can be used.

なお、上述した（１７）式に示す関数は、上述した（１３）式の条件を満たす。

Note that the function shown in the above-described equation (17) satisfies the condition of the above-described equation (13).

In addition to the above equation (17), for example, in the weighting function w (t), in the case of T ₀ ≦ T ≦ T ₁ , it is a continuous function that is convex upward, and otherwise Any function that can be zero can be applied.

  Here, in the above-described equations (6), (12), and (16), when the image coordinates (p, q) are outside the image, the pixel value I (p, q) is set to zero. (0) or zero vector.

  Further, when one or both of the components p and q of the image coordinates (p, q) are non-integer values, for example, the lattice point (p ′, q) that is closest to the image coordinates (p, q) The pixel value I (p ', q') of ') can be used as the pixel value I (p, q). Further, for example, the pixel value I (p, q) can be determined by interpolation processing from pixel values at a plurality of grid points around the image coordinates (p, q).

  Further, for example, the following equation (18) is used as a function for determining whether or not the image coordinates (p, q) are inside the image.

また、評価値Ｅは、上述した（６）式の代わりに、以下に示す（１９）式を用いることもできる。

Moreover, the evaluation value E can use the following equation (19) instead of the above-described equation (6).

これにより、画像内に存在するスキャン線分（の部分）の総延長で規格化された評価値Ｅを定義することができる。また、評価値Ｅは、同様に上述した（１２）式の代わりに、以下に示す（２０）式を用いてもよい。

Thereby, it is possible to define the evaluation value E normalized by the total extension of (parts of) scan line segments existing in the image. Similarly, the evaluation value E may use the following expression (20) instead of the above-described expression (12).

以上により、例えばサッカーコートのライン（白線の）ように、色の各成分が周囲よりもラインの方が高い場合において、上述した（７）式のｋ_Ｒ、ｋ_Ｇ、及びｋ_Ｂの各係数を０以上（但し、ｋ_Ｒ、ｋ_Ｇ、及びｋ_Ｂの何れか一つ以上は０より大きいものとする。例えば、ｋ_Ｒ＝０．３０、ｋ_Ｇ＝０．５９、及びｋ_Ｂ＝０．１１等）とし、更に、上述したＴ_０＝−１０、Ｔ_１＝１０とし、上述した（１５）式〜（１７）式を適用した場合には、カメラパラメータの校正が精密であればあるほど、上述した（１６）式の評価値Ｅは大きくなる。

As described above, for example, when each color component is higher than the surroundings, such as a soccer court line (white line), each coefficient of k _R , k _G , and k _B in the above equation (7) Is 0 or more (provided that one or more of k _R , k _G , and k _B is greater than 0. For example, k _R = 0.30, k _G = 0.59, and k _B = 0. .11 etc.), and further, when T ₀ = −10 and T ₁ = 10 described above and the above-described equations (15) to (17) are applied, the calibration of the camera parameters is accurate. As described above, the evaluation value E of the above-described equation (16) increases.

  According to the embodiment described above, the robustness and accuracy of evaluation can be improved by quantifying the success or failure of calibration based on image information. Specifically, the geometric feature of the pattern of the calibration figure can be projected onto the image plane, and the pixel value can be evaluated in the vicinity of the projected image of the geometric feature information on the image taken by the camera to be calibrated. Thereby, the pixel value information of the original image that is lost in the conventional method of extracting feature points and lines in advance can be effectively used, and the robustness and accuracy of the evaluation of the success or failure of the calibration can be improved.

<Image calibration evaluation program>
Here, the above-described image calibration evaluation apparatus 10 can perform the image calibration evaluation process according to the present invention by the above-described dedicated apparatus configuration. However, a part of the image calibration evaluation apparatus 10, for example, an image input unit, pattern storage, and the like. Each function of the means, the projecting means, the scan line segment setting means, the evaluation value calculating means, etc. may be realized by a computer.

  In this case, an execution program (image calibration evaluation program) for realizing each of the control functions described above is generated, and the execution program is installed in a computer such as a general-purpose personal computer or a server, for example. Processing can be realized.

  The execution program installed in the computer main body can be provided by a recording medium such as a CD-ROM. In this case, the recording medium in which the execution program is recorded is set in a drive device or the like provided in the computer, and the execution program included in the recording medium is installed from the recording medium to the auxiliary storage device or the like provided in the computer via the drive device. .

  As a recording medium, other than a CD-ROM, for example, a recording medium that records information optically, electrically, or magnetically, such as a flexible disk or a magneto-optical disk, a ROM (Read Only Memory), a flash memory, or the like As described above, various types of recording media such as a semiconductor memory for electrically recording information can be used.

  The computer also includes a network connection device that can be connected to a communication network, and obtains an execution program from another terminal connected to the communication network or the execution result obtained by executing the program or The execution program itself in the invention can be provided to other terminals.

  The auxiliary storage device provided in the computer is a storage means such as a hard disk, and can store the execution program in the present invention, a control program provided in the computer, and perform input / output as necessary. The memory device included in the computer stores an execution program read from the auxiliary storage device by the CPU. The memory device includes a ROM, a RAM (Random Access Memory), and the like.

  The computer also has a CPU (Central Processing Unit), and controls the overall processing of the computer, such as various operations and input / output of data between each component, based on a control program such as an OS (Operating System) and an execution program. Thus, each processing can be realized.

  As a result, the image calibration evaluation process can be efficiently realized at a low cost without requiring a special apparatus configuration. In addition, by installing the program, the image proofreading evaluation process can be easily realized.

<Image calibration evaluation process>
Next, an image proofreading evaluation processing procedure by the execution program in the present invention will be described using a flowchart.

<Image calibration evaluation processing procedure>
FIG. 8 is a flowchart showing an example of an image calibration evaluation processing procedure in the present embodiment. In FIG. 8, first, an image included in a video photographed by a camera to be calibrated is captured (S01). Next, a graphic feature projected on the image coordinate system is obtained based on a graphic feature (geometric feature) of a three-dimensional pattern including a straight line or a curve as a calibration graphic stored in advance and an input camera parameter ( S02).

  Next, in the above-described processing of S02, one or more points are set on the projected image of the calibration graphic on the image plane based on the projected graphic (S03), and each point passes through that point. A line segment is set as a scan line segment (S04).

  Next, in the image captured from the image input unit 11 by the processing of S01, the pixel value is multiplied by a preset weighting factor corresponding to the position on each scan line segment with respect to the pixel on the scan line segment group. On the other hand, the sum is obtained (S05), and an evaluation value obtained by quantifying the success / failure of the calibration based on the obtained sum is calculated and output by the above-described formulas (S06).

  Thereby, camera calibration evaluation can be reliably and quickly performed by numerical values. Further, according to the result, control such as feedback to various camera parameters of the camera to be calibrated can be performed.

<Evaluation Value E Calculation Processing Procedure>
Next, the evaluation value calculation processing procedure in the present embodiment in the processing of S06 described above will be specifically described with reference to a flowchart. FIG. 9 is a flowchart illustrating an example of an evaluation value calculation processing procedure in the present embodiment. In the following description, description will be made based on the above-described method for calculating the evaluation value E.

When calculating the evaluation value E, first, initialization (i = 0, E = 0) is performed (S11), and then the i th (where i∈ {0, 1, N−1}) is performed by the pattern storage unit 12. , N is an integer greater than or equal to 1), the coordinates (X _i , Y _i , Z _i ) and the tangential direction (U _i , V _i , W _i ) of the representative point are read (S12). Next, an image (x _i , y _i ) of the representative point is acquired from the coordinates (X _i , Y _i , Z _i ) and the above-described equation (2) (S13). The coordinates (X _i , Y _i , Z _i ), the tangential direction (U _i , V _i , W _i ), and the tangential image (u _i , v _i ) is acquired (S14).

Here, the value of j as a variable is initialized (j = 0) (S15), t _j = [T ₀ ] + j (S16), and the representative point image (x _i , y _i ) and the tangential direction A point (p _i (t _j ), q _i (t _j )) on the scan line segment is acquired based on the image (u _i , v _i ) and the above-described equation (4) (S17). The evaluation value _{E, E + w (t j)} f (I (p i (t j), q i (t j))) and to (S18), the variable j 1 increased (j = j + 1) ( S19) Then, it is determined whether or not the variable j is “floor [T ₁ ] −ceil [T ₀ ] +1” or more (S20). Here, when j is less than “floor [T ₁ ] −ceil [T ₀ ] +1” (NO in S20), the process returns to S16 and the process described later is performed. In the process of S20, when j is “floor [T ₁ ] −ceil [T ₀ ] +1” or more (YES in S20), i is increased by 1 (i = i + 1) (S21), i Is determined to be N or more (S22).

  Here, if i is less than N (NO in S22), the process returns to S12 and the processing described later is performed. If i is equal to or greater than N in the process of S22 (YES in S22), the evaluation value E is output (S23), and the process ends.

  Through the processing described above, the evaluation value E in the present embodiment can be calculated with high accuracy. Further, based on the evaluation value E obtained by the above-described processing, image calibration evaluation can be performed accurately and quickly, and control by feedback processing is performed in order to obtain an accurate video corresponding to the result. Highly accurate processing can be realized.

  As described above, according to the present invention, the robustness and accuracy of the evaluation can be improved by quantifying the success or failure of the calibration based on the image information. Specifically, the geometric feature (graphic feature) of the pattern for calibration is projected onto the image plane, and the pixel value is evaluated in the vicinity of the projected image of the geometric feature information on the image taken by the camera to be calibrated. Can do. Thereby, the pixel value information of the original image that is lost in the conventional method of extracting feature points and lines in advance can be effectively used, and the robustness and accuracy of the evaluation of the success or failure of the calibration can be improved.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows an example of a function structure of the image | video structure evaluation apparatus in this embodiment. It is a figure which shows an example of the camera pattern used for camera calibration. It is a figure which shows an example of the geometric feature information in this embodiment. It is a figure which shows an example of a scan line segment. It is a figure which shows an example of a weight function in case an argument is a continuous value. FIG. 6 is a diagram illustrating an example of a weight function when an argument different from FIG. 5 is a continuous value. It is a figure which shows an example of the weight function in case an argument is a discrete value. It is a flowchart which shows an example of the image calibration evaluation process sequence in this embodiment. It is a flowchart which shows an example of the calculation process procedure of the evaluation value in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image calibration evaluation apparatus 11 Image input means 12 Pattern storage means 13 Projection means 14 Scan line segment setting means 15 Evaluation value calculation means 21 Calibration pattern 31 Representative point 32 Tangent direction 41 Projected image 42 Scan line segment

Claims (6)

In an image calibration evaluation apparatus that evaluates the success or failure of image calibration in the imaging parameters of the imaging means from the image obtained by the imaging means,
Image input means for capturing an image to be calibrated imaged by the imaging means;
Pattern storage means for holding a graphic feature of a three-dimensional pattern including a straight line or a curve as a calibration graphic;
Projecting means for obtaining a graphic feature projected on an image coordinate system using the graphic feature and the input imaging parameter;
A scan line segment setting unit that sets one or more points on the projected image of the calibration graphic based on the graphic projected by the projection unit, and sets a line segment passing through each set point;
Evaluation value calculation means for obtaining a pixel value of each pixel on the line segment in the image captured by the image input means and calculating an evaluation value obtained by quantifying the success or failure of the calibration using the obtained pixel value; An image calibration evaluation apparatus characterized by comprising: The evaluation value calculation means includes:
An integral value or a sum is obtained by multiplying a pixel value of each pixel on all line segments obtained by the scan line segment setting means by a preset weighting factor corresponding to a position on each line segment. The image calibration evaluation apparatus according to claim 1. The projection means includes
Geometric feature information corresponding to the imaging parameter is obtained as the graphic feature from a plurality of geometric feature information stored in the pattern storage means, and the obtained geometric feature information is projected and converted to obtain geometric feature information in an image coordinate system. The image calibration evaluation apparatus according to claim 1, wherein: The graphic feature is:
4. One or more representative points on a pattern including a straight line or a curve obtained by the image pickup means, and a tangential direction of the straight line or the curve at the representative point are included. The image calibration evaluation apparatus according to 1. The scan line segment setting means includes
The image calibration evaluation apparatus according to claim 4, wherein a line segment that passes through an image of each representative point in the graphic feature and is orthogonal to an image in a tangential direction at the representative point is set as a scan line segment. In an image calibration evaluation program for evaluating the success or failure of image calibration in the imaging parameters of the imaging means, from an image obtained by the imaging means,
Computer
Image input means for capturing an image to be calibrated imaged by the imaging means;
Pattern storage means for holding graphic features of a three-dimensional pattern including a straight line or a curve as a calibration graphic;
Projecting means for obtaining a graphic feature projected on an image coordinate system using the graphic feature and the input imaging parameter;
Based on the figure projected by the projection means, one or more points are set on the projection image of the calibration figure, and a scan line segment setting means for setting a line segment passing through each set point; and
In the image captured by the image input unit, functions as an evaluation value calculation unit that acquires a pixel value of each pixel on the line segment and calculates an evaluation value obtained by quantifying the success or failure of the calibration using the acquired pixel value. An image calibration evaluation program characterized in that

Images