JP2010025569A - Camera parameter identification apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simply and correctly obtain camera parameters in comparison with a conventional apparatus. <P>SOLUTION: Two indexes A, B are installed at different locations within an image capture range by a camera on a reference plane such as the ground in a real space. Length dimensions of the indexes A, B are known and identical. The camera is installed so as to cause the horizontal direction of a captured image to be parallel to the horizontal direction of the real space. On this condition, a three-point perspective distortion is generated at the indexes A, B appearing on the captured image. A vertical vanishing point Pv and a horizontal vanishing point Ph can be identified by the three-point perspective distortion. The camera parameters including a focus length f and a depression angle ρ of the camera are obtained based on a distance β from the vertical vanishing point Pv to a Xi axis of an image coordinate system and a distance γ from the horizontal vanishing point Ph to the Xi axis. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a camera parameter specifying device, method, and program, and in particular, a focal length of a camera that is installed obliquely with respect to a reference plane and has a fixed shooting range, and an angle of a line of sight of the camera with respect to the reference plane. The present invention relates to a camera parameter specifying apparatus and method and a computer program for obtaining a camera parameter including:

  As a prior art regarding this type of technical field, for example, there is one disclosed in Patent Document 1. According to this prior art, first, a statistical characteristic value of a person's height is registered in advance. Then, the area including the ground is photographed by the camera, and a plurality of coordinate data of the landing part and the head of the pedestrian in the photographed image by the camera are collected. Furthermore, the plurality of coordinate data is substituted into a relational expression including the height of the pedestrian, the camera posture angle, the focal length of the camera (lens), and the position of the camera. You can get several. Then, statistical processing is performed on the plurality of relational expressions, so that an expression including the statistical characteristic value of the pedestrian's height is obtained, and the statistical characteristic value of the pedestrian's height in this expression is registered in advance. By substituting the statistical characteristic value of the height of a person, the relationship between the posture angle of the camera, the focal length of the camera, and the position of the camera can be obtained. Based on this relationship, one or more values of the camera attitude angle, the camera focal length, and the camera position are calculated.

Japanese Patent Laying-Open No. 2005-233846

  However, in the above-described prior art, in order to obtain camera parameters including the camera attitude angle, the camera focal length, and the camera position, the coordinate data of the landing part and the head of the pedestrian from the image captured by the camera. It is necessary to statistically process a plurality of relational expressions to which a plurality of coordinate data are substituted and a plurality of relational expressions to which the plurality of coordinate data are substituted. It is naturally troublesome to perform such processing (work), and it takes corresponding labor and time. Moreover, the formula for obtaining the camera parameter includes a statistical characteristic value of the height of the pedestrian, and the statistical characteristic value of the height of the person registered in advance as the statistical characteristic value of the height of the pedestrian. A characteristic value is substituted, that is, an estimated value is substituted. Accordingly, it is natural that an accurate camera parameter cannot be obtained depending on an expression in which such an estimated value is substituted.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a camera parameter characteristic apparatus and method and a computer program that can determine a camera parameter more easily and accurately than in the past.

  In order to achieve this object, the first invention of the present invention is applied to a camera installed so as to be inclined with respect to a reference plane and having a fixed photographing range. It is a camera parameter specifying device for obtaining camera parameters including a distance and an angle of the line of sight of the camera with respect to a reference plane. Specifically, the photographed image is acquired from the camera when an index capable of specifying the vertical vanishing point and the horizontal vanishing point with respect to the reference plane is present in the photographing range by the camera on the photographed image by the camera. Image acquisition means is provided. Then, based on the captured image acquired by the image acquisition means, a position specifying means for specifying the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional area including the captured image is also provided. In addition, a parameter calculation means for obtaining camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point specified by the position specifying means is also provided.

  That is, in the first invention, the camera is set so as to be inclined with respect to the reference plane, and the shooting range of the camera is fixed. Then, a photographed image when the index is present within the photographing range by the camera is obtained from the camera by the image obtaining means. On the captured image acquired by the image acquisition means, it is possible to specify the vertical vanishing point and the horizontal vanishing point with respect to the reference plane. For this reason, the indicator has, for example, a portion that extends in a direction perpendicular to the reference plane and that forms a plurality of first straight lines that are parallel to each other at a distance from each other. On the captured image, the images appear non-parallel to each other due to so-called three-point perspective distortion. A point where the extended lines of the first straight lines (images) that appear to be non-parallel to each other converge is defined as a vertical vanishing point. In addition, the indicator has a portion that extends in a direction parallel to the reference plane and forms a plurality of second straight lines that are parallel to each other at a distance from each other. They appear non-parallel to each other on the captured image. A point where the extended lines of the second straight lines that appear to be non-parallel to each other converge is defined as a horizontal vanishing point. Based on such a photographed image, the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional region including the photographed image are identified by the position identifying means. Then, based on the positions of the vertical vanishing point and the horizontal vanishing point, the parameter calculation means obtains camera parameters.

  Note that the camera is preferably installed so that the horizontal direction of the captured image is parallel to the reference plane. In this case, a vertical vanishing point exists on a straight line that passes through the center of the captured image and extends along the vertical direction of the captured image. Thereby, at least the position of the vertical vanishing point can be easily specified, and hence the camera parameter can be easily specified.

  On the other hand, when the horizontal direction of the captured image is not parallel to the reference plane, specifically, when the camera is rotating (that is, rolling) around the line of sight, the following may be performed. That is, a declination calculation means for obtaining a declination with respect to a reference plane in the horizontal direction of the captured image, and a vertical vanishing point when it is assumed that the declination is zero based on the declination obtained by the declination calculation means And position virtual means for virtualizing the position of each horizontal vanishing point. Then, the parameter calculation means may obtain the camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point virtualized by the position virtual means.

  In the first invention, a two-dimensional image coordinate system may be set in a two-dimensional area including a captured image. In this case, the position specifying means specifies the position of each of the vertical vanishing point and the horizontal vanishing point in the two-dimensional area with the coordinate values of the image coordinate system.

  Further, when a three-dimensional world coordinate system is set in the three-dimensional space including the imaging range, the three-dimensional space in which at least one of the three element values constituting the coordinate value of the world coordinate system is known. A reference distance setting means for setting a distance between any two points as a reference distance, and a camera position for obtaining a position of the camera in a three-dimensional space including a distance from the reference plane to the camera based on the set reference distance And an arithmetic means. Then, the camera position is added to the camera parameter, and based on the camera parameter, a coordinate value in the other corresponding to an arbitrary coordinate value in one of the image coordinate system and the world coordinate system is obtained, that is, the image coordinate system and the world You may further provide the coordinate conversion means which performs mutual conversion between coordinate systems.

  According to this configuration, the position of the camera including the distance from the reference plane to the camera is added to the camera parameter. By using this camera parameter, mutual conversion between the image coordinate system and the world coordinate system, so-called coordinate conversion, becomes possible.

  Even if the position of the camera, particularly the distance from the reference plane to the camera, is unknown, the world coordinate system has no scale, that is, an arbitrary unit coordinate system by considering this as one constant. Can be handled. In such an arbitrary unit world coordinate system, it is not clear what scale (unit) each coordinate value is based on, but for example, the distance between two points and the distance between two points. The ratio can be determined. Specifically, the ratio of the height of one building to another building in the captured image is calculated, or the height of a certain person is estimated compared to the height of a certain building. Further, the distance traveled by a moving object such as a person can be obtained by comparing with the distance between any two points.

  Here, it is desirable that the world coordinate system is set so that one of the three coordinate axes constituting the world coordinate system extends perpendicularly to the reference plane. It is desirable that the camera is installed on a coordinate axis extending perpendicular to the reference plane. Furthermore, at least one of the two arbitrary points that define the reference distance described above has a coordinate axis value extending perpendicularly to the reference plane among the three element values constituting the coordinate value in the world coordinate system. It is desirable to be known. Strictly speaking, under this condition, the position of the camera including the distance from the reference plane to the camera is obtained.

  Subsequently, the second invention of the present invention is a method invention corresponding to the first invention. That is, the present invention is applied to a camera that is installed obliquely with respect to a reference plane and whose shooting range is fixed, and includes a focal length of the camera and an angle of the line of sight of the camera with respect to the reference plane. This is a camera parameter specifying method for obtaining a parameter. Specifically, the photographed image is acquired from the camera when an index capable of specifying the vertical vanishing point and the horizontal vanishing point with respect to the reference plane is present in the photographing range by the camera on the photographed image by the camera. An image acquisition process is provided. A position specifying process for specifying the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional region including the photographed image based on the photographed image obtained in the image obtaining process is also provided. Furthermore, a parameter calculation process for obtaining camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point specified in the position specifying process is also provided.

  A third invention of the present invention is an invention relating to a computer program corresponding to the first invention. That is, the present invention is applied to a camera that is installed obliquely with respect to a reference plane and whose shooting range is fixed, and includes a focal length of the camera and an angle of the line of sight of the camera with respect to the reference plane. This is a camera parameter specifying program for obtaining parameters. Specifically, the photographed image is acquired from the camera when an index capable of specifying the vertical vanishing point and the horizontal vanishing point with respect to the reference plane is present in the photographing range by the camera on the photographed image by the camera. The computer executes the image acquisition procedure. Further, based on the captured image acquired by the image acquisition procedure, a position specifying procedure for specifying the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional region including the captured image, and the position specifying procedure. The computer also executes a parameter calculation procedure for obtaining camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point.

  As described above, according to the present invention, the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional region including the photographed image are specified based on the photographed image when the index exists within the photographing range of the camera. Then, camera parameters are obtained based on the positions of the vertical vanishing point and the horizontal vanishing point. In other words, the above-mentioned conventional technique needs to collect a plurality of coordinate data of a pedestrian's landing part and head from a photographed image of a camera and perform statistical processing on a plurality of relational expressions into which the plurality of coordinate data are substituted. Unlike the above, such processing (work) is not required. Unlike the related art, the estimated value is not substituted into the equation for obtaining the camera parameter. Therefore, camera parameters can be obtained more easily and accurately than in the past.

  An embodiment of the present invention will be described by taking the camera calibration system 10 shown in FIG. 1 as an example.

As shown in the figure, a camera calibration system 10 according to the present embodiment includes a personal computer (hereinafter referred to as a PC) 12, and the PC 12 has a fixed photographing range (field of view). A camera (hereinafter simply referred to as a camera) 14 is connected. The PC 12 functions as a camera calibration device that calibrates the camera 14 by executing a camera calibration program installed in itself. The camera calibration program is a known CD-ROM (Compact Disk Read Only
Memory) and DVD (Digital
The recording medium 16 is supplied via an appropriate recording medium 16 such as a Versatile Disk. Further, the PC 12 is attached with a command input device 18 for inputting various commands. Examples of the command input device 18 include a mouse 20 and a keyboard 22, but other devices may be used. Further, the PC 12 is also provided with, for example, a liquid crystal display 24 as display means, and various information including an image taken by the camera 14 is displayed on the display 24 as will be described later.

  As shown in FIG. 2, the camera 14 is installed in a real space with the horizontal ground 30 as a reference plane. Specifically, the lens principal point Oc is placed at a position of a height η from the ground 30, and the line of sight (optical axis) 32 is inclined with respect to the ground 30. It is installed so that the direction is parallel to the horizontal direction of real space. Then, when camera calibration is performed, two indexes A and B are placed on the ground 30 and within the shooting range of the camera 14. These indices A and B are objects that extend vertically upward from the ground 30 respectively, and their respective length dimensions Da and Db are known and are the same (Da = Db). As such indexes A and B, for example, humans having the same height are suitable. These indices A and B are the lens principal points Oc when projected from a lens principal point Oc of the camera 14 onto a distance (strictly speaking, a vertical plane including the line of sight 32 of the camera 14 (a plane perpendicular to the ground 30). Are installed at positions different from each other, that is, a vertical vanishing point Pv and a horizontal vanishing point Ph described later can be specified.

  Here, camera calibration in the present embodiment refers to obtaining the focal length f, depression angle ρ, and installation height η of the camera 14 as camera parameters. The focal length f of the camera 14 is the distance from the lens principal point Oc to the imaging surface Cs of the imaging device 34 in the camera 14, and the depression angle ρ is the angle of the line of sight 32 with respect to the horizontal plane (ground 30). The installation height η is a distance from the ground 30 to the lens principal point Oc as described above. In the camera calibration in the present embodiment, the horizontal field angle θh and the vertical field angle θv of the camera 14 are also obtained as camera parameters. Such camera calibration is realized in the following manner.

  That is, it is assumed that a photographed image as shown in FIG. 3, for example, is obtained in the state shown in FIG. 2, that is, displayed on the display 24 described above. Note that the lattice pattern 36 in FIG. 3 indicates that the indicators A and B are arranged at different positions on the ground 30 and that the horizontal direction of the captured image is parallel to the horizontal direction of the real space. This is a fictitious thing that indicates that is installed. As can be seen from FIG. 3, the indicators A and B (images) on the photographed image are all slanted (so long as they exist at positions off the center in the horizontal direction of the photographed image) due to so-called three-point perspective distortion. And are displayed with different sizes (as long as they exist at different positions in the vertical direction of the captured image).

  In such a captured image, a two-dimensional image coordinate system is set as shown in FIG. Specifically, the center Oi of the captured image is set as the origin. Then, the Xi axis is set so as to extend through the origin Oi and along the horizontal direction of the photographed image, and the Yi axis is set so as to be orthogonal to the Xi axis at the origin Oi. This image coordinate system is set not only on the photographed image but also in a range outside the photographed image, that is, in a wide two-dimensional area including the photographed image. In this image coordinate system, the intersection Pv between the straight line 38 passing through the vertex Pua and the bottom point Pda of one index A and the straight line 40 passing through the vertex Pub and the bottom point Pdb of the other index B is specified. Is done. This intersection Pv is a so-called vertical vanishing point and exists on the Yi axis, which is the vertical axis of the image coordinate system. In addition, the intersection Ph of the straight line 42 passing through the vertices Pua and Pub of each index A and B and the straight line 44 passing through the bottom points Pda and Pdb of each index A and B are specified. This intersection Ph is a horizontal vanishing point, and exists on the opposite side of the vertical vanishing point Pv across the Xi axis that is the horizontal axis of the image coordinate system. Then, based on the image coordinate values (Xi, Yi) of these vanishing points Pv and Ph, specifically, based on the distances β and γ from the vanishing points Pv and Ph to the Xi axis, the focal length f and The depression angle ρ is obtained, and further, the installation height η, the horizontal field angle θh, and the vertical field angle θv are obtained.

  For this purpose, first, as shown in FIG. 5, a lens principal point Oc of the camera 14 and an equivalent imaging surface Cs ′ that is conjugate (point-symmetric) with the imaging surface Cs described above with respect to the lens principal point Oc, Consider the mutual positional relationship between the indices A and B. As described above, since each of the indexes A and B extends vertically upward from the ground 30, the extending direction of the index A (line segment PuaPda) and the extending direction of the index B (line) Min (PubPdb) are parallel to each other. Here, as shown in FIG. 6, a plane Sa including the vertex Pua and the bottom point Pda of one index A and the lens principal point Oc is assumed. In addition, a plane C obtained by extending the equivalent imaging surface Cs ′ is assumed, and a straight line La that is an intersection line of the plane C and the plane Sa is assumed. This straight line La represents a straight line passing through the vertex Pua and the bottom point Pda of the index A (image thereof) projected onto the equivalent imaging plane Cs ′ as viewed from the lens principal point Oc, that is, corresponds to the straight line 38 shown in FIG. To do.

  Similarly, as illustrated in FIG. 7, a plane Sb including the vertex Pub and the bottom point Pdb of the other index B and the lens principal point Oc is assumed. A straight line Lb that is an intersection line between the plane Sb and the plane C described above is assumed. This straight line Lb represents a straight line passing through the vertex Pub and the bottom point Pdb of the index B (image) projected onto the equivalent imaging surface Cs ′ when viewed from the lens principal point Oc, that is, corresponds to the straight line 40 shown in FIG. To do.

  Further, as shown in FIG. 8, a straight line V that is an intersection line of the plane Sa in FIG. 6 and the plane Sb in FIG. 7 is assumed. Here, the plane Sa includes the vertex Pua and the bottom point Pda of the index A and the lens principal point Oc, and the plane Sb includes the vertex Pub and the bottom point Pdb of the index B and the lens principal point Oc. Since the extending directions of the indices A and B are parallel to each other, the straight line V passes through the lens principal point Oc and extends in the extending directions of the indices A and B (line segment PuPda and line segment PubPdb). Become parallel.

  Then, as shown in FIG. 9, an intersection point Qv between the straight line V and the plane C is assumed. Here, since both the straight line V and the straight line La shown in FIG. 6 exist on the plane Sa and are not parallel to each other, they always cross somewhere. In addition, since the straight line La exists on the plane C, the straight line La and the straight line V intersect on the plane C. Furthermore, the plane C intersects with the straight line V only at the point Qv. When these are put together, as shown in FIG. 10, the straight line La and the straight line V cross each other at a point Qv. Similarly, the straight line Lb and the straight line V cross each other at the point Qv.

  That is, the straight line La and the straight line Lb intersect at a point Qv on the straight line V. As described above, since the straight line La corresponds to the straight line 38 shown in FIG. 4 and the straight line Lb corresponds to the straight line 40 shown in FIG. 4, the point Qv corresponds to the vertical disappearance shown in FIG. Corresponds to point Pv.

  Subsequently, as shown in FIG. 11, a line segment Lu (PuaPub) connecting the vertices Pua and Pub of each index A and B and a line segment Ld (PdaPdb) connecting the base points Pda and Pdb are obtained. Suppose. As described above, the extending directions of the indicators A and B are parallel to each other, and the lengths Da and Db of the indicators A and B are the same as each other. Ld has the same length dimension as each other and is parallel to each other.

  Here, as shown in FIG. 12, a plane Ta including both ends Pua and Pub of one line segment Lu and the lens principal point Oc is assumed. Furthermore, a straight line Ma that is an intersection line between the plane Ta and the plane C is assumed. This straight line Ma represents a straight line passing through both ends Pua and Pub of the line segment Lu (image thereof) projected onto the equivalent imaging surface Cs ′ when viewed from the lens principal point Oc, that is, corresponds to the straight line 42 shown in FIG. .

  Similarly, as shown in FIG. 13, a plane Tb including both ends Pda and Pdb of the other line segment Ld and the lens principal point Oc is assumed. Further, a straight line Mb that is an intersection line between the plane Tb and the plane C is assumed. This straight line Mb represents a straight line passing through both ends Pda and Pdb of the line segment Ld (image thereof) projected onto the equivalent imaging surface Cs ′ as viewed from the lens principal point Oc, that is, corresponds to the straight line 44 shown in FIG. .

  Then, an intersection line between the plane Ta in FIG. 12 and the plane Tb in FIG. 13 is assumed. The plane Ta includes both ends Pua and Pub of the line segment Lu and the lens principal point Oc, and the plane Tb includes both ends Pda and Pdb of the line segment Ld and the lens principal point Oc, and each line segment Lu. Since Ld and Ld are parallel to each other, the intersection line is a straight line passing through the lens principal point Oc and parallel to the respective line segments Lu and Ld, as indicated by reference numeral H in FIG.

  Furthermore, as shown in FIG. 14, an intersection point Qh between the straight line H and the plane C is assumed. Here, since both the straight line H and the straight line Ma shown in FIG. 12 exist on the plane Ta and are not parallel to each other, they always cross somewhere. In addition, since the straight line Ma exists on the plane C, the straight line Ma and the straight line H intersect on the plane C. The plane C intersects with the straight line H only at the point Qh. When these are combined, as shown in FIG. 14, the straight line Ma and the straight line H cross each other at a point Qh. Similarly, the straight line Mb and the straight line H also intersect each other at the point Qh.

  That is, the straight line Ma and the straight line Mb intersect with each other at the point Qh on the straight line H. Since the straight line Ma corresponds to the straight line 42 shown in FIG. 4 and the straight line Mb corresponds to the straight line 44 shown in FIG. 4, as described above, the straight line Qh is the horizontal disappearance shown in FIG. Corresponds to point Ph.

  With these things in mind, as shown in FIG. 15, a three-dimensional world coordinate system composed of three orthogonal axes such as an Xw axis, a Yw axis, and a Zw axis is defined.

  That is, as shown in the figure, the Xw axis and the Yw axis are set on the ground 30 in the real space. In short, a two-dimensional plane (Xw-Yw plane) composed of the Xw axis and the Yw axis hits the ground 30. Then, at the origin Ow that is the intersection of these Xw axis and Yw axis, the Zw axis is set to be orthogonal to each of the Xw axis and Yw axis. That is, the Zw axis is a vertical axis that defines the height in real space. Although not shown in the figure, the indicators A and B are arranged on the Xw-Yw plane that is the ground 30 so as to extend parallel to the Zw axis.

  In this world coordinate system, the camera 14 is described in detail so that the lens principal point Oc is located on the Z axis and the line of sight 32 is inclined with respect to the Xw-Yw axis that is the ground 30. The extension line is installed so that the upper end edge and the lower end edge of the equivalent imaging plane Cs ′ are parallel to the Xw axis so that the extension line obliquely intersects the Yw axis. According to this condition, the straight line V described above coincides with the Zw axis. Accordingly, the world coordinate values (Xw, Yw, Zw) of the lens principal point Oc are (0, 0, η). Further, assuming a straight line 46 that is an intersection of a two-dimensional plane (Yw-Zw plane) composed of the Yw axis and the Zw axis and a plane C including the equivalent imaging plane Cs ′, this straight line 46 is equivalent to the equivalent imaging plane. It passes through the center Os of Cs ′, intersects the Yw axis at a non-right angle, and intersects the Zw axis at a non-right angle.

  On the other hand, since the equivalent imaging plane Cs ′ is conjugate with the imaging plane Cs, the image coordinate system shown in FIG. 4 can be converted onto the equivalent imaging plane Cs ′. That is, as shown in FIG. 16, the center Os of the equivalent imaging surface Cs ′ is set as the origin Oi of the image coordinate system. Then, the Xi axis of the image coordinate system is set so as to pass through the origin Oi and be parallel to the upper edge and the lower edge of the equivalent imaging surface Cs ′, and the image coordinates are orthogonal to the Xi axis at the origin Oi. The system Yi axis is set. In FIG. 16 as well, the image coordinate system is set not only on the equivalent imaging surface Cs ′ but also in a wide range on the plane C including the equivalent imaging surface Cs ′.

  As described above, the image coordinate system set on the plane C including the equivalent imaging surface Cs ′ is superimposed on the world coordinate system shown in FIG. The vertical vanishing point Pv is associated. That is, as shown in FIG. 17, the Yi axis of the image coordinate system coincides with a straight line 46 that is an intersection line of the plane C and the Yw-Zw plane in the world coordinate system. Here, as described with reference to FIG. 10, the plane C intersects with the straight line V only at the vertical vanishing point Pv, and the straight line V coincides with the Zw axis of the world coordinate system as described above. The vertical vanishing point Pv corresponds to the intersection of the straight line 46 that is the Yi axis of the image coordinate axis and the Zw axis of the world coordinate system. Therefore, the world coordinate value (Xw, Yw, Zw) of this vertical vanishing point Pv is expressed as (0, 0, Pvz) using an unknown number, for example, Pvz. On the other hand, in the image coordinate system, the coordinate value (Xi, Yi) of the vertical vanishing point Pv is (0, −β) as shown in FIG. 18 using the distance β shown in FIG. . Then, in the world coordinate system, the distance Pv−Os from the vertical vanishing point Pv to the center Os of the equivalent imaging plane Cs ′ is determined by the image coordinate value (0, −β) of the vertical vanishing point Pv, that is, β and Become.

  Similarly, the horizontal vanishing point Ph will be considered. That is, as shown in FIG. 19, a plane R passing through the lens principal point Oc and parallel to the Xw-Yw plane (ground 30) is assumed. Then, the straight line H shown in FIG. 14 exists on this plane R. Since the horizontal vanishing point Ph exists on the straight line H, the horizontal vanishing point Ph also exists on the plane R. Accordingly, the world coordinate value (Xw, Yw, Zw) of the horizontal vanishing point Ph is expressed as (Phx, Phy, η) using two unknowns, for example, Phx and Phy. Further, a straight line 48 that is an intersection line between the plane R and the plane C is assumed. Here, the Yi axis of the image coordinate system exists on the plane C, and this Yi axis is not parallel to the Xw-Yw plane, so the straight line 48 intersects the Yi axis. At the same time, since the horizontal vanishing point Ph exists on the plane R and also on the plane C, the horizontal vanishing point Ph exists on the straight line 48. Then, the intersection Ph ′ between the straight line 48 and the Yi axis can be considered as the horizontal vanishing point Ph moved along the plane R to the position of Xw = 0. The world coordinate value (Xw, Yw, Zw) of the intersection Ph ′ is expressed as (0, Phy, η). On the other hand, in the image coordinate system, the coordinate value (Xi, Yi) of the horizontal vanishing point Ph is (Phi, γ) as shown in FIG. 20, for example, using the distance γ shown in FIG. . The image coordinate value (Xi, Yi) of the intersection Ph ′ is (0, γ). Therefore, in the world coordinate system, the distance Ph′−Os from the intersection Ph ′ to the center Os of the equivalent imaging surface Cs ′ is determined by the image coordinate value (0, γ) of the intersection Ph ′, that is, γ.

  Arranging the positional relationships of the main parts shown in FIG. 17 and FIG. 19 results in FIG. 21 and FIG. FIG. 21A is a view of the relevant parts viewed from the front end side of the Zw axis, and FIG. 21B is a view of the relevant parts viewed from the front end side of the Xw axis. FIG. 22A is a view of the relevant parts as viewed from the above-described intersection Ph ′ side (the tip side of the Yi axis), and FIG. 22B is the same as FIG. 21B. In FIGS. 21 and 22, the distance f ′ from the lens principal point Oc to the equivalent imaging surface Cs ′ is equivalent to the focal length f. The effective horizontal dimension Dh of the equivalent imaging surface Cs ′ is equivalent to that of the actual imaging surface Cs, and the unit is [pixel (pixel)]. The effective vertical dimension Dv of the equivalent imaging surface Cs ′ is also equivalent to that of the actual imaging surface Cs, and its unit is [pixel]. Furthermore, at least one of the effective horizontal dimension Dh and the effective vertical dimension Dv, for example, the effective horizontal dimension Dh is known. The aspect ratio Dh: Dv, which is the ratio of both Dh and Dv, is also known, for example, 4: 3.

  According to the positional relationship shown in FIGS. 21 and 22, it can be seen that the distances β and γ are expressed by the following equations 1 and 2, respectively.

  Then, when each of these formulas 1 and 2 is transformed into an equation for tan ρ, the following formula 3 is derived.

  Further, when the formula 3 is transformed into an expression for the focal length f, the following formula 4 for obtaining the focal length f is derived. Note that the unit of the focal length f represented by this equation 4 is [pixel].

  Then, by dividing the number 2 by the above number 1, the following number 5 is derived. When the number 5 is transformed into an expression for the depression angle ρ, the expression 6 for obtaining the depression angle ρ is derived.

  In particular, from the positional relationship shown in FIG. 22A, it can be seen that the horizontal angle of view θh is expressed by the following equation (7).

  Then, the vertical angle of view θv is expressed by the following Expression 8 using the aspect ratio Dh: Dv of the imaging surface Cs (equivalent imaging surface Cs ′).

  That is, according to the present embodiment, the focal length f of the camera 14 is obtained based on Equation 4, and the depression angle ρ is obtained based on Equation 6. For this reason, on the captured image shown in FIG. 4, the vertices Pua and Pub and the bottom points Pda and Pdb of the indices A and B are specified by, for example, operating the mouse 20 shown in FIG. Then, the image coordinate values (Xi, Yi) of each of the identified points Pua, Pub, Pda and Pdb, for example (xiu, yuua) for the point Pua, (xida, yida) for the point Pda, and the point Pub (Xiub, yiub), and for the point Pdb, (xidb, yidb), the vanishing points Pv and Ph are identified. In short, the image coordinate values (0, -β) of the vanishing points Pv and Ph And (Phi, γ) (or (0, γ)). Further, based on the image coordinate values (0, −β) and (Phi, γ) of the vanishing points Pv and Ph, the distances β and γ are obtained, and the distances β and γ are expressed by the above-described formula 4 and By substituting into Equation 6, the focal length f and the depression angle ρ are obtained. Then, the horizontal angle of view θh of the camera 14 is obtained based on Equation 7, and the vertical angle of view θv is obtained based on Equation 8. Further, the installation height η of the camera 14 is obtained as follows.

  Referring to FIG. 23, in the world coordinate system, a vector Vua (xvua, yvua, zvua) from the lens principal point Oc of the camera 14 toward one of the indices A and B, for example, the vertex Pua of the index A, is obtained. . Specifically, a vector Ve (xe, ye, ze) heading from the lens principal point Oc to an arbitrary point in the world coordinate system uses the image coordinate value (Xi, Yi) corresponding to the arbitrary point as follows. It is obtained by Equation 9.

  Accordingly, the vector Ve (xe, ye, ze) obtained by substituting the image coordinate value (xiua, iuua) of the vertex Pua of the index A into the image coordinate value (Xi, Yi) in the equation 9 is the lens. The vector Vua (xvua, yvua, zvua) is directed from the principal point Oc to the vertex Pua of the index A.

  Similarly, a vector Vda (xvda, yvda, zvda) from the lens principal point Oc toward the bottom point Pda of the index A is obtained. That is, the vector Ve (xe, ye, ze) obtained by substituting the image coordinate value (xida, yida) of the bottom point Pda of the index A into the image coordinate value (Xi, Yi) in Equation 9 is the lens. The vector Vda (xvda, yvda, zvda) is directed from the principal point Oc to the bottom point Pda of the index A.

  Here, it is assumed that the distance (element value Yw) in the Yw-axis direction from the origin Ow of the world coordinate system to the index A is δ. At the same time, the distance (element value Xw) in the Xw-axis direction from the origin Ow to the index A is ignored (Xw = 0). Then, based on the distance δ, the installation height η, the length dimension Da of the index A, and the vector Vua (xvua, yvua, zvaa) from the lens principal point Oc toward the vertex Pua of the index A, Derived.

  Then, in this equation 10, when attention is paid to the distance δ, the following equation 11 is derived.

  Similarly, the following equation 12 is derived based on the distance δ, the installation height η, and the vector Vda (xvda, yvda, zvda) from the lens principal point Oc toward the bottom point Pda of the index A.

  In the equation (12), when attention is paid to the distance δ, the following equation (13) is derived.

  Further, since the equations 11 and 13 are equivalent to each other, the relationship represented by the following equation 14 is established.

  Then, by transforming this equation 14 into an expression for the installation height η, the following equation 15 for obtaining the installation height η is derived.

  That is, according to this equation 15, the vector Vua (xvua, yvaa, zvaa) from the lens principal point Oc toward the vertex Pua of one index A and the vector Vda (xvda, yvda, The installation height η is determined based on zvda) and the length dimension Da of the index A. The vector Vua (xvaa, yvua, zvua) toward the vertex Pua is obtained from the image coordinate values (xiua, yuua) of the vertex Pua, and the vector Vda (xvda, yvda, zvda) toward the base point Pda is obtained. As described above, it is obtained from the image coordinate values (xida, yida) of the bottom point Pda.

  Note that the installation height η may be obtained by paying attention to the other index B instead of one index A. That is, based on the image coordinate values (xib, yib) and (xidb, yidb) of the vertex Pub and the bottom point Pdb of the other index B, the respective vectors Vub (xvub, yvub, zvub) and Vdb (xvdb, yvdb, zvdb) may be obtained, and the installation height η may be obtained based on these vectors Vub (xvub, yvub, zvub) and Vdb (xvdb, yvdb, zvdb) and the length dimension Db of the index B. .

  In order to realize this series of camera calibration, the PC 12 (strictly, a CPU (not shown) in the PC 12) shown in FIG. 1 operates as follows according to the camera calibration program.

  That is, with reference to FIG. 24, first, in step S1, the PC 12 acquires a captured image for calibration as shown in FIG. 3, specifically, a still image for one frame. The acquisition of the image follows a command given from the command input device 18, that is, a manual operation using the mouse 20 and the keyboard 22.

  Then, the PC 12 proceeds to step S3, specifies the vertex Pua and the bottom point Pda of one index A, that is, obtains the respective image coordinate values (xiua, iuua) and (xida, yida). This specification also follows a command given from the command input device 18, for example, according to a click operation of the mouse 20.

  Further, the PC 12 proceeds to step S5, and acquires the length dimension Da of the one index A. The length dimension Da is given from the command input device 18, for example, from the keyboard 22.

  Then, the PC 12 proceeds to step S7 and specifies the vertex Pub and the bottom point Pdb of the other index B in the same manner as step S3 described above, that is, the respective image coordinate values (xib, iuub) and (xidb, yidb). ) And it progresses to step S9 and the length dimension Db of the said other parameter | index B is acquired in the same way as the above-mentioned step S5.

  After executing step S9, the PC 12 proceeds to step S11, specifies the vanishing points Pv and Ph, that is, obtains the respective image coordinate values (0, −β) and (Phi, γ). In step S13, the distances β and γ are obtained based on the image coordinate values (0, −β) and (Phi, γ).

  Further, the PC 12 proceeds to step S15, obtains the focal length f of the camera 14 based on the above-described equation 4, and then proceeds to step S17 to obtain the depression angle ρ based on the above-described equation 6. Then, the process proceeds to step S19, and after obtaining the horizontal angle of view θh based on the above equation 7, the process proceeds to step S21, and the vertical angle of view θv is obtained based on the above equation 8.

  After executing step S21, the PC 12 proceeds to step S23. In step S23, the installation height η of the camera 14 is obtained by the procedure including the above-described equations 9 and 15. With the execution of step S23, the PC 12 ends a series of camera calibrations.

  By using the camera parameters obtained by this series of camera calibrations, mutual conversion between the coordinate values (Xi, Yi) of the image coordinate system and the coordinate values (Xw, Yw, Zw) of the world coordinate system becomes possible.

  Specifically, for the world coordinate values (Xw, Yw, Zw) corresponding to the arbitrary image coordinate values (Xi, Yi), the vector Ve (xe, ye, ze) represented by the above equation 9 is used. It is obtained by the following equation (16).

  In Equation 16, ε is a coefficient, and this coefficient ε is uniquely determined if any one of the three element values Xw, Yw and Zw constituting the world coordinate value (Xw, Yw, Zw) is determined. . In this embodiment, since the Xw-Yw plane is set on the ground 30, that is, Zw = 0, the coefficient ε relating to the world coordinate value (Xw, Yw, 0) of an arbitrary point on the ground 30 is It is uniquely determined by Equation 17.

  Therefore, by substituting Equation 17 into Equation 16, mutual conversion between the image coordinate value (Xi, Yi) and the world coordinate value (Xw, Yw, 0) under the condition of Zw = 0 becomes possible.

  Further, based on the correspondence relationship between the image coordinate value (Xi, Yi) and the world coordinate value (Xw, Yw, 0) under the condition of Zw = 0, the world coordinate value (Xw, Yw, 0) is passed. In addition, mutual conversion between world coordinate values (Xw, Yw, Zw ′) of arbitrary points extending in a direction perpendicular to the ground 30 and corresponding image coordinate values (Xi ′, Yi ′) is also possible. It is.

  Such camera calibration is suitable for monitoring applications, for example, for recognizing an absolute position, size, moving distance, etc. in a real space such as a moving object projected on a captured image from the captured image. . Further, based on the recognition result, for example, it is possible to perform control such as tracking the moving object with a turning camera.

  As described above, according to the present embodiment, the world coordinate system is set as shown in FIG. 15, and the image coordinate system is set as shown in FIG. 16 (FIG. 4). Then, two indicators A and B are set within the shooting range by the camera 14, and the two vanishing points Pv specified by the two indicators A and B projected on the shot image causing a three-point perspective distortion. And the image coordinate values (0, −β) and (Phi, γ) of Ph, camera parameters including the focal length f, depression angle ρ, and installation height η of the camera 14 are obtained, that is, camera calibration is performed. Realized. Therefore, it is necessary to collect a plurality of coordinate data of the landing part and the head of the pedestrian from the photographed image of the camera and to perform statistical processing on a plurality of relational expressions into which the plurality of coordinate data are substituted. Unlike the above, such processing (work) is not required. Unlike the related art, the estimated value is not substituted into the equation for obtaining the camera parameter. Therefore, camera parameters can be obtained more easily and accurately than in the past. This becomes more significant as the number of cameras 14 to be calibrated increases.

  In the present embodiment, the lengths Da and Db of the indices A and B are the same as each other, but the present invention is not limited to this. For example, it is assumed that the length dimension Da of one index A is smaller than the length dimension Db of the other index B (Da <Db). Then, it is assumed that the indices A and B are displayed on the captured image with the sizes Da ′ and Db ′ corresponding to the respective length dimensions Da and Db, as shown in FIG. In this case, when it is assumed that the length dimensions Da and Db of the indices A and B are the same, a temporary index that will be displayed on the captured image, for example, the length dimension Db of the larger index B is set. A provisional index B ′ that matches the length dimension Da of the smaller index A is assumed. The length dimension Db ″ of the temporary index B ′ is obtained by the following equation (18).

  Then, the vanishing points Pv and Ph are identified based on the provisional index B ′ having the corrected length dimension Db ″ and the index A in which the length dimension Da is unchanged. In this way, accurate camera calibration is realized even under conditions where the lengths Da and Db of the indices A and B are different from each other (Da ≠ Db). Conversely, the length dimension Da of the smaller index A may be matched with the length dimension Db of the larger index B. However, in terms of accuracy, the length dimension Db of the larger index B is smaller. It is desirable to match the length A of the index A.

  In the present embodiment, the upper and lower edges of the imaging surface Cs (equivalent imaging surface Cs ′) of the camera 14 are parallel to the Xw axis of the world coordinate system, that is, the horizontal direction of the captured image is real space. Although the camera 14 is installed so as to be parallel to the horizontal direction, the present invention is not limited to this. That is, when the horizontal direction of the photographed image is not parallel to the horizontal direction of the real space, in short, when the camera 14 is rotated (that is, rolled) around the line of sight 32, as shown in FIG. It exists at a position away from the Yi axis of the coordinate system. In this case, of course, the horizontal vanishing point Ph also exists at an inappropriate position, and accurate camera calibration cannot be realized in this state. However, when the camera 14 is rolling, the equivalent imaging surface Cs ′ described above should be in a state as originally indicated by the dotted line 50 in FIG. 27, and 1 in FIG. It is in a rotating state as indicated by a dashed line 52. That is, the positional relationship between the center Os of the equivalent imaging surface Cs ′ and the respective indices A and B is unchanged regardless of whether the camera 14 is rolled. Therefore, when the camera 14 is rolling, the following measures may be taken.

  First, as shown in FIG. 28, a straight line 54 passing through the origin Oi of the image coordinate system and the vertical vanishing point Pv is assumed. Then, an angle α formed by the straight line 54 and the Yi axis of the image coordinate system is obtained. This angle α corresponds to the deviation angle in the horizontal direction of the captured image with respect to the horizontal direction of the real space. Then, the vertical vanishing point Pv is rotated by the declination α by the rotation determinant expressed by the following equation 19 using the declination α. That is, the image coordinate value (Xi ″, Yi ″) of the vertical vanishing point Pv ″ when the argument α is assumed to be α = 0 is assumed.

  Similarly, the horizontal vanishing point Ph is also rotationally moved by the deviation angle α according to this equation 19. In other words, the image coordinate value (Xi ″, Yi ″) of the horizontal vanishing point Ph ″ when the argument α is assumed to be α = 0 is virtualized. As shown in FIG. Based on the image coordinate values (Xi ″, Yi ″) of the vanishing points Pv ″ and Ph ″, the distances β and γ can be obtained, and thus camera calibration can be realized. In this way, the camera 14 rolls. Even under the circumstances, accurate camera calibration is realized.

  Furthermore, for each index A and B, separate objects (human beings) may not be used, but one common object may be used as each index A and B. Specifically, as shown in FIG. 30A, when a certain object is present at a certain place (time ta), the object (the image thereof) in the captured image is used as an index A, and the same object moves. The object in the captured image when it exists in another place (time tb) is defined as an index B. Then, camera calibration may be performed using two indexes A and B based on the same object photographed at different times ta and tb.

  Further, as the indicators A and B, humans are preferable as described above, but are not limited thereto. For example, two rod-like bodies having the same lengths Da and Db may be adopted as the respective indexes A and B. Of course, even if the lengths Da and Db are not the same, according to the procedure described with reference to FIG. 25, accurate camera calibration can be realized. Furthermore, an existing structure having a straight portion such as a utility pole or a building may be adopted as each of the indicators A and B. Further, instead of the two indexes A and B, one rectangular frame (or plate) 60 as shown in FIG. 31 may be adopted. Specifically, two opposite sides of the four sides of the frame 60 are perpendicular to the ground 30 not shown in the figure, and the other two sides are parallel to the ground 30 (or the ground 30), the frame body 60 may be installed. In short, the vanishing points Pv and Ph described above may be formed, and as one means for that purpose, in this embodiment, two indexes A and B are adopted.

  Further, in the present embodiment, as described with reference to FIG. 24, the vertices Pua and Pub and the bottom points Pda and Pdb of each index A and B are manually operated to specify the vanishing points Pv and Ph. However, it is not limited to this. For example, by using an appropriate feature extraction method such as the Hough transform method for extracting a straight line portion, the straight line portions such as the indices A and B are automatically extracted, and the vanishing points Pv and Ph are automatically extracted. You may make it specify. In addition, when each of the indicators A and B is a moving body such as a human, the moving body as each of the indicators A and B is automatically detected using an appropriate moving body detection method such as a background difference method. As a result, the vanishing points Pv and Ph may be automatically specified.

  Of the camera parameters, the horizontal field angle θh and the vertical field angle θv do not need to be obtained unless there is a special need. In extreme cases, the installation height η of the camera 14 may not be accurately obtained.

  That is, in the conversion equation shown in Expression 16, when Zw = 0, the Expression 16 is expressed as the following Expression 20 using the above Expression 17.

  According to this equation 20, even if the exact installation height η is unknown, by considering this as one constant, the world coordinate values (Xw, Yw, Zw) in arbitrary units having no scale can be obtained. can get. That is, it is unknown what scale (unit) the world coordinate value (Xw, Yw, Zw) itself is based on, but even in a world coordinate system without the scale, a distance between any two points (for example, The ratio of the length of the line segment is known. Therefore, an accurate value of the installation height η is not necessary in an application where it is sufficient to know the ratio of the distances. For example, the ratio of the height of one building to another in the captured image is obtained, the height of a person is estimated relative to the height of the building, and the person has moved The distance can be obtained by comparing with the distance between any two points.

  Further, even if the installation height η is not an accurate value, if a relatively approximate value is set, a relatively accurate world coordinate value (Xw, Yw, Zw) can be obtained. Further, in the above-described equation 15 for obtaining the installation height η, even if an approximate value is set instead of an accurate value as the length dimension Da of the index A, a relatively accurate installation height is obtained. Then, η is obtained, and as a result, relatively accurate world coordinate values (Xw, Yw, Zw) are obtained. The same applies to the case where the installation height η is obtained based on the length dimension Db of the index B.

  Furthermore, in Equation 16, not only when Zw = 0, but if any one of the element values Xw, Yw and Zw of the world coordinate values (Xw, Yw, Zw) is known, the world coordinate value Mutual conversion between (Xw, Yw, Zw) and image coordinate values (Xi, Yi) is possible. For example, when Xw is known, the equation 16 is expressed as the following equation 21. In other words, mutual conversion is possible by this equation (21).

  When Yw is known, mutual conversion is possible by the following equation 22, and when Zw is known, mutual conversion is possible by equation 23.

  Further, if these formulas 21 to 23 are used, any two points Pa and Pb in which at least one of the three element values Xw, Yw, and Zw constituting the world coordinate values (Xw, Yw, Zw) is known are known. The distance λ between them can be determined.

  Specifically, the world coordinate value (Xw, Yw, Zw) of one point Pa is (Xwa, Ywa, Zwa), and the world coordinate value (Xw, Yw, Zw) of the other point Pb is (Xwb, Ywb, Zwb). Then, the distance λ between these two points Pa and Pb is expressed by the following equation (24).

  Here, for example, it is assumed that the element value Xwa of the Xw axis is known for one point Pa, and the element value Xwb of the Xw axis is also known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 21, and the world coordinate value (Xwb, Ywb, Zwb) is also substituted as the world coordinate value (Xw, Yw, Zw) of the equation 21 and further substituted into the equation 24, that is, the distance between the two points Pa and Pb by the following equation 25 λ is determined.

  Next, it is assumed that the element value Xwa of the Xw axis is known for one point Pa, and the element value Ywb of the Yw axis is known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 21, and the world coordinate value (Xwb, Ywb, Zwb) is substituted as the world coordinate value (Xw, Yw, Zw) of the equation (22) and further substituted into the equation (24), that is, according to the following equation (26), the distance λ between the two points Pa and Pb becomes Desired.

  Further, it is assumed that the element value Xwa of the Xw axis is known for one point Pa, and the element value Zwb of the Zw axis is known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 21, and the world coordinate value (Xwb, Ywb, Zwb) is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 23, and these are substituted into Equation 24, that is, the following Equation 27 obtains the distance λ between the two points Pa and Pb. It is done.

  Subsequently, it is assumed that the element value Ywa of the Yw axis is known for one point Pa, and the element value Ywb of the Yw axis is also known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 22, and the world coordinate value (Xwb, Ywb, Zwb) is also substituted as the world coordinate value (Xw, Yw, Zw) of Formula 22, and by substituting these into Formula 24, that is, by the following Formula 28, the distance λ between the two points Pa and Pb is Desired.

  It is assumed that the element value Ywa on the Yw axis is known for one point Pa, and the element value Zwb on the Zw axis is known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 22, and the world coordinate value (Xwb, Ywb, Zwb) is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 23, and these are substituted into Equation 24, that is, the following Equation 29 yields the distance λ between the two points Pa and Pb. It is done.

  Further, it is assumed that the element value Zwa of the Zw axis is known for one point Pa, and the element value Zwb of the Zw axis is also known for the other point Pb. In this case, the world coordinate value (Xwa, Ywa, Zwa) of one point Pa is substituted as the world coordinate value (Xw, Yw, Zw) of Equation 23, and the world coordinate value (Xwb, Ywb, Zwb) is also substituted as the world coordinate value (Xw, Yw, Zw) of Equation 23, and by substituting these into Equation 24, that is, by the following Equation 30, the distance λ between the two points Pa and Pb is Desired.

  As can be seen with reference to these formulas 25 to 30, the installation height η is not included in formulas 25, 26, and 28. That is, according to these formulas 25, 26, and 28, the distance λ between the two points Pa and Pb is obtained even if the installation height η is unknown. On the other hand, for the equations 27, 29, and 30, the installation height η is included. That is, according to these equations 27, 29, and 30, the distance λ between the two points Pa and Pb is uniquely determined by determining the installation height η.

  Here, as one application example, a case will be described in which it is assumed that an elongated object is arranged on the ground surface 30 so as to extend in the vertical direction, and the length dimension λ of the object is obtained. In this case, one end of the object is handled as the point Pa and the other end is handled as the point Pb.

  For example, it is assumed that the element value Xwa of the Xw axis is already known for one end Pa of the object. Then, since the object extends in the vertical direction, naturally, the element value Xwb of the Xw axis is also known for the other end Pb, that is, Xwa = Xwb. In this case, the length dimension λ of the object is obtained based on the above-described equation 25, more specifically, based on the following equation 31 in which Xwa = Xwb in the equation 25.

  On the other hand, for example, if the element value Ywa of the Yw axis is known for one end Pa of the object, since the object extends in the vertical direction, naturally, the other end Pb also has Yw. The axis element value Ywb is known, that is, Ywa = Ywb. In this case, the length dimension λ of the object is obtained based on the above equation 28, more specifically, based on the following equation 32 in which Ywa = Ywb in the equation 28.

  Furthermore, it is assumed that the element value Zwa of the Zw axis is known for one end Pa of the object. In this case, follow the procedure below. That is, the following equation 33 is obtained by substituting the world coordinate value (Xwa, Ywa, Zwa) of one end Pa of this object as the above-described equation 23 (Xw, Yw, Zw). Further, the world coordinate value (Xwa, Ywa, Zwa) of the other end Pb is substituted as the world coordinate value (Xw, Yw, Zw) of the above-described equation 16, thereby obtaining the equation 34.

  When attention is paid to these equations 33 and 34, since Xwa = Xwb, the following equation 35 is established. Similarly, since Ywa = Ywb, Expression 36 is established.

  Then, by substituting one of these formulas 35 and 36, for example, formula 35 into the above formula 24, the following formula 37 is obtained. Similarly, Expression 36 is obtained by substituting Expression 36 into Expression 24.

  In other words, the length dimension λ of the object is obtained based on either of these formulas 37 and 38. In addition, in these equations 37 and 38, unlike the above equations 31 and 32, the installation height η of the camera 14 is required.

  Further, paying attention to the above-described equation 30, when the equation 30 is, for example, Zwa = Zwb = 0, the equation 30 is expressed as the following equation 39.

  In the equation (39), the element values xe, ye and ze of the vector Ve (xe, ye, ze) included in the square root are obtained by the above equation (9). The number 39 is expressed as the following number 40.

  Further, in this equation 40, when λ = 1, and when the equation 40 is re-transformed into an expression for the height dimension η, the following equation 41 is derived.

  That is, the number 41 represents the installation height η with the distance λ between the two points Pa and Pb as a unit. Therefore, if the height η represented by this equation 41 is used, a scale in units of the distance λ between the two points Pa and Pb can be given to the world coordinate system. For example, the distance between any other two points can be obtained using the distance λ as a unit.

  The above-mentioned number 40 is conditional on Zwa = Zwb = 0, but when Zwa = Zwb = Zw ′ ≠ 0, the following number 42 holds.

  Then, in this equation 42, when λ = 1, the equation 42 is derived by re-transforming the equation 42 into an expression for the height dimension η.

  That is, even when Zwa = Zwb = Zw ′ ≠ 0, a scale having the unit of the distance λ between the two points Pa and Pb can be given to the world coordinate system.

  The contents described in the above embodiments are merely specific examples for realizing the present invention, and are not intended to limit the present invention. For example, instead of paying attention to the above-described equation 30, the equation representing the installation height η may be obtained with the distance λ between the two points Pa and Pb as a unit by paying attention to the equations 27 and 29.

  The camera calibration program is supplied by an appropriate recording medium 16, but may be supplied through an electric communication line such as the Internet.

  Further, instead of realizing the camera calibration device by causing the so-called general-purpose PC 12 to execute the camera calibration program, a dedicated camera calibration device specialized for camera calibration may be realized.

  Further, the present invention may be applied not only to the camera calibration but simply to determine one of the focal length f, the depression angle ρ, and the installation height η of the camera 14. The focal length f, depression angle ρ, and installation height η can be determined with a certain degree of accuracy when the camera 14 is installed, but the accuracy depends on the ability of the operator (human). Therefore, the present invention is extremely effective in obtaining the focal length f, the depression angle ρ, and the installation height η more accurately.

It is a figure showing the schematic structure of the camera calibration system concerning one embodiment of the present invention. It is an illustration figure which shows the positional relationship of a camera and each parameter | index in the embodiment. It is an illustration figure which shows an example of the picked-up image with a camera in the embodiment. It is an illustration figure for demonstrating the calculation procedure of a camera parameter in the embodiment. It is an illustration figure which shows the mutual positional relationship of each principal part of a camera and each parameter | index in the same embodiment. It is an illustration figure for demonstrating the same positional relationship in order. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure following FIG. It is an illustration figure for demonstrating the definition of a world coordinate system in the embodiment. It is an illustration figure for demonstrating the definition of an image coordinate system in the embodiment. It is an illustration figure which shows the relationship between a world coordinate system and an image coordinate system in the embodiment. It is the illustration figure which paid its attention to the image coordinate system in FIG. FIG. 18 is an illustrative view different from FIG. 17 showing the relationship between the world coordinate system and the image coordinate system in the same embodiment. It is the illustration figure which paid its attention to the image coordinate system in FIG. FIG. 20 is an illustrative view in which the relationships shown in FIGS. 17 and 19 are arranged. FIG. 20 is an illustrative view different from FIG. 21 in which the relationships shown in FIGS. 17 and 19 are arranged. It is an illustration figure for demonstrating the procedure which calculates the installation height of a camera among the camera parameters in the embodiment. 4 is a flowchart illustrating an overview of a camera calibration program executed by a PC in the embodiment. It is an illustration figure for demonstrating the coping method when the length dimension of each parameter | index differs in the same embodiment. It is an illustration figure which shows an example of the picked-up image when the camera is rolling in the embodiment. It is an illustration figure which shows the concept of the same roll. It is an illustration figure for demonstrating the coping method when the roll has arisen. It is an illustration figure following FIG. It is an illustration figure which shows an example which covers each parameter | index with one object in the same embodiment. It is an illustration figure which shows an example of another parameter | index in the same embodiment.

Explanation of symbols

10 Camera calibration system 12 PC
14 Camera 16 CPU
18 Command input device 24 Display 30 Ground A, B Index Oc Lens principal point f Focal length ρ Depression angle

Claims (7)

A camera parameter specifying device that obtains camera parameters including a focal length of a camera that is installed obliquely with respect to a reference plane and whose shooting range is fixed, and an angle of a line of sight of the camera with respect to the reference plane,
An image acquisition means for acquiring, from the camera, an image capable of specifying a vertical vanishing point and a horizontal vanishing point with respect to the reference plane on the image captured by the camera within the imaging range;
Position specifying means for specifying the positions of the vertical vanishing point and the horizontal vanishing point in a two-dimensional region including the photographed image based on the photographed image obtained by the image obtaining means;
Parameter computing means for obtaining the camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point identified by the position identifying means;
A camera parameter specifying device. The camera was installed so that the horizontal direction of the captured image was parallel to the reference plane,
The camera parameter specifying device according to claim 1. The camera is installed so that the horizontal direction of the captured image is not parallel to the reference plane, and under these conditions,
A declination calculating means for obtaining a declination with respect to the reference plane in the horizontal direction of the captured image;
Position virtual means for virtualizing the positions of the vertical vanishing point and the horizontal vanishing point in the two-dimensional region when the declination is assumed to be zero based on the declination;
Further comprising
The parameter calculation means obtains the camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point virtualized by the position virtual means,
The camera parameter specifying device according to claim 1. A two-dimensional image coordinate system is set in the two-dimensional region,
The position specifying means specifies the position of each of the vertical vanishing point and the horizontal vanishing point by a coordinate value in the image coordinate system;
The camera parameter specifying device according to claim 1. A three-dimensional world coordinate system is set in the three-dimensional space including the shooting range,
Reference distance setting means for setting, as a reference distance, a distance between any two points in the three-dimensional space in which at least one of three element values constituting coordinate values in the world coordinate system is known;
Camera position calculation means for determining the position of the camera in the three-dimensional space including the distance from the reference plane to the camera based on the reference distance;
Coordinate conversion means for obtaining a coordinate value in the other corresponding to an arbitrary coordinate value in one of the image coordinate system and the world coordinate system based on the camera parameters including the position of the camera;
The camera parameter specifying device according to claim 4. A camera parameter specifying method for obtaining a camera parameter including a focal length of a camera installed to be inclined with respect to a reference plane and having a fixed shooting range and an angle of a line of sight of the camera with respect to the reference plane,
An image acquisition process for acquiring, from the camera, an image capable of specifying a vertical vanishing point and a horizontal vanishing point with respect to the reference plane on the image captured by the camera within the imaging range;
A position specifying process for specifying the positions of the vertical vanishing point and the horizontal vanishing point in a two-dimensional region including the photographed image based on the photographed image obtained in the image obtaining process;
A parameter calculation process for obtaining the camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point identified in the position identifying process;
A camera parameter specifying method comprising: A camera parameter specifying program for obtaining camera parameters including a focal length of a camera that is installed obliquely with respect to a reference plane and a shooting range is fixed, and an angle of a line of sight of the camera with respect to the reference plane,
An image acquisition procedure for acquiring, from the camera, the captured image having an index capable of specifying a vertical vanishing point and a horizontal vanishing point with respect to the reference plane on the captured image by the camera;
A position specifying procedure for specifying the position of each of the vertical vanishing point and the horizontal vanishing point in a two-dimensional region including the photographed image based on the photographed image obtained in the image obtaining procedure;
A parameter calculation procedure for obtaining the camera parameters based on the positions of the vertical vanishing point and the horizontal vanishing point identified by the position identifying procedure;
Is a camera parameter specifying program that causes a computer to execute.

Images