JP2003194520A - Camera apparatus and system and method for calibrating camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera apparatus that can automate camera calibrating work. <P>SOLUTION: A camera 200 photographs a board 100 for calibration on which a reference position 101, a surrounding pattern, a central pattern 103, and corner patterns 104 are formed from a prescribed position. A camera calibrating device 300 correlates the screen coordinate of the reference position 101 with a world coordinate based on the image of the board 100 photographed by means of the camera 200, calculates the calibrating parameter of the camera 200 based on the correlated result, and calibrates the camera 200 in accordance with the calculated calibrating parameter. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera device, a camera calibration system and a camera calibration method for inspecting and monitoring by imaging.

[0002]

2. Description of the Related Art In recent years, a camera calibration system has been developed in the industrial field for obtaining the distance and direction from a camera to an object to be imaged for inspection and monitoring. The camera is calibrated by arranging a calibration board on which a pattern of reference positions is formed at a predetermined position and capturing an image of the calibration board with the camera. Here, the reference position pattern is a pattern indicating a reference position for camera calibration. As the pattern of the reference position, a pattern in which feature points can be easily extracted visually is usually used.

FIG. 12 is a diagram showing a calibration board used in a conventional camera calibration system. The calibration board 400 has a pattern of reference positions made of squares arranged equidistant from each other, and is imaged by a camera arranged at a predetermined position. The user uses an input means such as a mouse to input the coordinates of the square apex on the captured image as a reference position. The coordinates of this vertex are expressed by two-dimensional coordinates on the xy plane (hereinafter referred to as screen coordinates). The coordinates of the vertices may be automatically obtained by pattern recognition.

After that, the screen coordinates of the vertices of the square are associated with the actual three-dimensional coordinates in the xyz space (hereinafter referred to as world coordinates). Such correspondence between the screen coordinates and the world coordinates is performed on a plurality of vertices, and the parameters for camera calibration (hereinafter referred to as camera calibration parameters) are calculated by inputting these coordinates.

For details of this process, refer to "(A Versatile C
amera Calibration Technique for High-Accuracy 3D Ma
chine Vision Metrology Using Off-the-Shelf TV Came
rasand Lenses ", Roger Y. Tsai, IEEE Journal of Rob
otics and automation Vol. RA-3. No. 4 August 1987,
pp323-344).

[0006]

However, in the above-mentioned conventional camera calibration system, the pattern of the reference position formed on the calibration board 400 shows only the reference position, and the calibration board 400 is not shown. There has been a problem in that it is necessary to manually associate the screen coordinates and the world coordinates with the above plurality of reference positions, and the camera calibration work cannot be automated.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a camera device, a camera calibration system, and a camera calibration method that can automate camera calibration work.

[0008]

In order to achieve the above object, a camera device of the present invention is a camera device for performing calibration according to a calibration parameter, and a reference position indicating a reference position for calibration and the reference. Imaging means for imaging a calibration imaging object at a predetermined position on which identification information for identifying a position is formed, and a calibration parameter for calculating the calibration parameter based on the image of the calibration imaging object. And a calculating means. With this configuration, the camera calibration work can be automated.

Further, the calibration image pickup object is provided with a plurality of enclosing patterns for grouping the plurality of reference positions. With this configuration, the reference positions can be grouped at the stage of pattern recognition, and the position fixing process of the reference positions can be simplified.

Further, the plurality of surrounding patterns include a position information surrounding pattern indicating position information in world coordinates as the identification information. With this configuration, all position information necessary for calibration can be acquired with reference to the position information of the position information enclosing pattern.

Further, the calibration imaging object is provided with a plurality of surfaces on which the plurality of surrounding patterns are aligned in parallel. With this configuration, since the grouped reference positions are aligned, the position identification process between groups can be simplified.

Further, the plurality of surrounding patterns include, as the identification information, a plurality of connecting surrounding patterns in which a positional relationship between the surrounding patterns is defined. Among the plurality of connecting surrounding patterns, A position information enclosing pattern indicating position information in world coordinates depending on the number of the reference positions and the arrangement of the reference positions is included. With this configuration, even if the image of the calibration imaging object is distorted and the reference position is not on the straight line, the screen coordinates and the world coordinates can be associated with each other.

Further, the connection surrounding pattern is a linear connection surrounding pattern in which a pattern defining a positional relationship between the surrounding patterns is formed by a straight line.
With this configuration, since the pattern is formed in a straight line, it is possible to easily perform pattern formation by printing or engraving.

Further, the connecting surrounding pattern is a curved connecting surrounding pattern in which a pattern defining a positional relationship between the surrounding patterns is formed by a curved line. With this configuration, it is possible to reduce the size of the surrounding pattern, increase the number of reference positions in the same area, and reduce the size of the calibration imaging target.

A reference position for changing a position information curve connection surrounding pattern, which is included as one of the curve connection surrounding patterns and indicates position information in world coordinates according to the number of the reference positions and the arrangement of the reference positions. Is provided in the position information curve connection surrounding pattern. With this configuration, the imaging range of the camera can be changed to an arbitrary position of the calibration imaging object.

Further, the calibration parameter calculation means, reference position coordinate calculation means for calculating the screen coordinates of the center of gravity of the reference position pattern for each of the surrounding patterns,
An origin identification unit that extracts position information from the reference position in the position information enclosing pattern, a surface dividing unit that classifies the reference position for each surface, and a reference position that is classified for each surface and the enclosing pattern in the row direction and Based on the reference position alignment means arranged in the column direction, the coordinate correspondence means for associating the screen coordinates with the world coordinates based on the aligned reference positions, and the correspondence relationship between the screen coordinates and the world coordinates, the calibration And a calculating means for calculating the parameter for use. With this configuration, the calibration parameter can be calculated by automatically associating the screen coordinates of the reference position with the world coordinates.

The camera calibration system of the present invention comprises a calibration imaging object on which a reference position indicating a reference position for calibration and identification information for specifying the reference position are formed.
A camera for imaging the calibration imaging object from a predetermined position, and a calibration parameter of the camera is calculated based on the captured image of the calibration imaging object, and the calibration parameter is calculated according to the calculated calibration parameter. And a camera calibration device for calibrating the camera. With this configuration, the camera calibration work can be automated.

According to the camera calibration method of the present invention, a camera is used to capture an image of a calibration imaging object on which a reference position indicating a reference position for calibration and identification information for identifying the reference position are formed. And a step of calculating a calibration parameter of the camera based on the captured image of the calibration imaging object, and a step of calibrating the camera according to the calculated calibration parameter. And By this procedure, the camera calibration work can be automated.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a block diagram showing the arrangement of a camera calibration system according to the first embodiment. The camera calibration system according to the present embodiment includes the calibration board 10
0, a camera 200 to be calibrated, and a camera calibration device 300. Here, the camera calibration device 3
00 may be housed in the camera body or may be provided as a device separate from the camera body. As will be described later, the camera calibration device 300 calculates camera calibration parameters based on an image obtained by capturing the calibration board 100 with the camera 200.

FIG. 2 is a block diagram showing the configuration of the camera calibration device 300. The camera calibration device 300 includes a reference position coordinate calculation unit 310, an origin identification unit 320, and a surface division unit 33.
0, reference position alignment unit 340, coordinate correspondence unit 350, and parameter calculation unit 360. The reference position coordinate calculation unit 310 calculates the screen coordinates of the center of gravity of the reference position pattern 101 for each surrounding pattern 102. The origin identification unit 320 extracts position information from the reference position calculated by the reference position coordinate calculation unit 310. Surface dividing unit 330
Categorizes the reference positions for each surface from the surfaces where the surrounding patterns 102 are aligned in parallel. The reference position alignment unit 340 sets the reference positions classified for each surface / enclosed pattern in the line direction in the unit of surface,
Align the reference positions in the column direction. The coordinate correspondence unit 350 associates the screen coordinates of the reference position with the world coordinates. The camera parameter calculation unit 360 obtains a camera calibration parameter from the correspondence between the screen coordinates of the reference position and the world coordinates. In the present embodiment, each of these units is realized by a CPU (not shown) in the camera calibration device executing a program stored in a ROM (not shown). Of course, instead of using the CPU and the ROM, it may be realized by a dedicated circuit for hardware.

On the other hand, the calibration board 100 has a reference position for camera calibration and identification information for specifying the reference position, as will be described later, and the camera 20
It is arranged at a predetermined position in front of 0 and is imaged by the camera 200. FIG. 3 is a diagram showing the calibration board 100.
The calibration board 100 has a three-dimensional structure,
It has a front surface pattern 110 located in front of the camera 200, a left side surface pattern 120 located to the left of the camera 200, a right side surface pattern 130 located to the right of the camera 200, and a bottom surface pattern 140 located below the camera 200. The corner pattern and the central pattern (see FIG. 4) described later are formed at the recessed positions in the figure.

FIG. 4 is a diagram showing a surface pattern. The same figure (A) shows the front surface pattern 110, and the same figure (B) shows the left side surface pattern 120, the right side surface pattern 130, and the bottom surface pattern 140. The front pattern 110 includes a reference position pattern (simply referred to as a reference position) 10 indicating a reference position.
1, a plurality of columns of enclosing patterns 102 that group a predetermined number (here, 12) of reference positions, and a position information enclosing pattern that is identification information for specifying the reference position is formed as other enclosing patterns. ing. This position information surrounding pattern is composed of a central pattern 103 and corner patterns 104 located on both sides of the central pattern 103.

On the other hand, the left side pattern 120, the right side pattern 130, and the bottom surface pattern 140 each have a reference position pattern 101 indicating a reference position, and a plurality of columns for grouping a predetermined number (here, 12) of reference positions. Of the surrounding pattern 102 of FIG. In the figure, the dotted line indicating the pattern is drawn to indicate that the pattern is inside, and does not actually exist.

The operation of the camera calibration system having the above configuration will be described. FIG. 5 is a flowchart showing a camera calibration processing procedure. First, the reference position coordinate calculation unit 310 calculates the screen coordinates of the centers of gravity of the reference position patterns 101 grouped by the surrounding pattern 102 (step S1). This processing is performed using a known pattern processing technique. As a result of the calculation, the screen coordinates grouped for each enclosing pattern are obtained as follows.

Surrounding pattern 0 = {screen position 00 of the reference position, ...
..., screen coordinates of reference position 0m0} Enclosing pattern 1 = {screen coordinates of reference position 10, ...
..., screen coordinate of reference position 0m1}, ... enclosing pattern n = {screen position of reference position n0, ...
..., screen coordinates of reference position 0mn} With this, the reference positions can be grouped for each row at the stage of pattern recognition, so that the process of determining the position of the reference position can be simplified.

The origin identifying section 320 extracts position information from the position information surrounding pattern (center pattern 103, corner pattern 104) (step S2). In this process, first, the surrounding pattern 102 including two reference positions is extracted as a candidate for the corner pattern 104. FIG. 6 is a diagram showing the origin identification processing and the surface division processing. A straight line a is obtained by connecting the reference positions 101 in the central pattern 103 and the corner pattern 104. Further, a straight line b is obtained by connecting the reference positions 101 in the lowermost surrounding pattern 106 of the left side surface pattern 120. Furthermore, the front pattern 11
A straight line c is obtained by connecting the reference position 101 at the left end of the enclosing pattern 102 of 0.

Central pattern 103 and corner pattern 10
Since the straight line a passing through the reference positions of 4 should be aligned on a straight line, based on the two reference positions in the corner patterns 104 extracted as candidates, the surrounding pattern 102 whose reference position is a straight line A pattern 104 and a central pattern 103 are used. Corner pattern 104 and central pattern 1
Since the world coordinates of the reference position 03 are known, the world coordinates of the reference positions of the other surrounding patterns 102 are relatively determined. In this way, all position information can be acquired with reference to the position information of the position information surrounding pattern.

The surface dividing section 330 is used for all surrounding patterns 1.
02 is classified into a front surface pattern 110, a left side surface pattern 120, a right side surface pattern 130, and a bottom surface pattern 140 (step S3). First, the surrounding pattern 106 at the bottom of the side surface is extracted from the corner pattern 104. As shown in FIG. 6, the intersection of the straight line a of the corner pattern 104 and the straight line b of the surrounding pattern 106 at the bottom of the side surface is the corner pattern 104.
If it is on the left side, it should be the shortest with the reference position 101 on the left side, so that the surrounding pattern 106 at the bottom of the side surface is extracted. Thus, the straight line a and the straight line b can be used to classify the front / left and right side surrounding patterns and the bottom surrounding pattern.

The screen coordinates of the reference positions of all the enclosing patterns are arranged in x and y coordinates for each enclosing pattern. Using the reference position 101 of the left corner pattern 104 and the two leftmost reference positions 101 of the surrounding pattern 102 in the front and other areas, these three points can be regarded as a straight line c
To extract. This process is performed on the right side as well, and a similar straight line is extracted. With these straight lines c, the enclosing patterns on the left side surface, the right side surface, and the front surface are classified. Then, the reference position aligning unit 340 uses the screen coordinates of the reference position 101 for each surface to sequentially align the reference positions 101 in the row direction and the column direction (step S4).

The coordinate correspondence unit 350 associates the screen coordinates calculated by the reference position coordinate calculation unit 310 with the world coordinates for each reference position (step S5). FIG. 7 is a diagram showing a correspondence relationship between screen coordinates and world coordinates.
The x-coordinate, y-coordinate, and z-coordinate are set with the left corner of the bottom pattern 140 as the origin of world coordinates. In FIG. 7, only the reference positions of the front pattern 110 and the bottom pattern 140 are drawn for simplicity. For example, the bottom pattern 14
Consider the world coordinates of the reference position 101 at the position of 4 rows and 5 columns of 0. Here, if all the intervals of the reference positions 101 are 50 mm, the world coordinates are (5 × 5
0,4 × 50,0), that is, (250,200,0).

The parameter calculation unit 360 includes a coordinate correspondence unit 3
The camera calibration parameters are calculated using the screen coordinates and the world coordinates associated in step 50 (step S
6). Formula (1) shows the relationship between world coordinates and camera coordinates. Here, Xw, Yw, and Zw represent world coordinates. x,
y and z represent camera coordinates. r1 to r9 represent rotation matrices of coordinates. Tx, Ty, and Tz represent moving matrices of coordinates.

[0032]

[Equation 1]

Xf = f · x / z (2), Yf = f · y / z (3) Equations (2) and (3) show the relationship between camera coordinates and screen coordinates. Here, Xf and Yf represent screen coordinates. f represents the focal length.

Using the above equations (1), (2), and (3), the rotation matrix, the movement matrix, and the focal length, which are the camera calibration parameters, are calculated from the correspondence between the screen coordinates of a plurality of reference positions and the world coordinates. calculate. Camera calibration device 30
In 0, the camera 200 is calibrated based on the calculated camera calibration parameter, and this processing ends.

As described above, according to the camera calibration system of the first embodiment, the camera calibration parameters are calculated by automatically associating the screen coordinates and the world coordinates of the plurality of reference positions with each other. You can
Camera calibration work can be automated.

(Second Embodiment) The camera calibration system of the second embodiment is different from the first embodiment only in the reference position and the surrounding pattern formed on the calibration board 100. , And other configurations are the same.

FIG. 8 is a diagram showing a linear connection surrounding pattern according to the second embodiment. In the second embodiment,
Two types of linear connection enclosing patterns 5 on the calibration board 100
00, 510, and position information linear connection surrounding pattern 520 having position information.

FIG. 9 is a diagram showing a processing procedure of a linear connection surrounding pattern. The same figure (A) is a linear connection surrounding pattern 50.
0B, and FIG. 3B shows the linear connection surrounding pattern 510 and the position information linear connection surrounding pattern 520. The linear connection surrounding pattern is classified into a linear connection surrounding pattern 500 having four reference positions 600 to 630, a linear connecting surrounding pattern 510 having two reference positions, and position information linear connecting surrounding pattern 520 having five reference positions. It Surrounding patterns with other numbers of reference positions are excluded as pattern recognition errors.

First, as shown in FIG. 9A, the center of gravity 640 of the reference position is obtained from the screen coordinates of the four reference positions included in the linear connection surrounding pattern 500. A reference position is set to 0 by appropriately selecting a reference position. The angle formed by the straight line connecting the reference position 0 and the center of gravity 640 and each reference position is defined as the reference position 1, the reference position 2, and the reference position 3 in order from the smallest. In the position information linear connection / surrounding pattern 520 having five reference positions, after the center of gravity 640 is obtained, the reference position closest to the center of gravity 640 is not processed. Further, a line segment passing through all the adjacent reference positions is created. In FIG. 1B, a line segment a passing through the reference positions 0 and 3 and a line segment b passing through the reference positions 0 and 1 are shown.

A reference position on the created line segment or within a certain distance from the line segment is searched from the reference positions in the linear connection surrounding pattern 510 having two reference positions. At this time, when the reference position can be retrieved, the line segment and the reference position are considered to be connected and stored. In this way, the connection states of all the linear connection surrounding patterns 500, 510 are checked. Then, by using the position information linear connection / surrounding pattern 520 as a starting point and following the stored connection, the positional relationship of the linear connection / surrounding pattern can be examined. Finally, the world coordinates of the reference position are determined in order from the known world coordinates of the linear connection surrounding pattern.

According to the second embodiment, the screen coordinates and the world coordinates can be associated with each other even if the image on the calibration board is distorted and the reference position is not on a straight line. Further, since the pattern is formed in a straight line, pattern formation by printing or engraving can be easily performed.

(Third Embodiment) In the camera calibration system of the third embodiment, as in the second embodiment, the camera calibration system is formed on the calibration board 100 as compared with the first embodiment. The other configurations are the same except for the reference position and the surrounding pattern.

FIG. 10 is a diagram showing a curve connection enclosing pattern in the third embodiment. The calibration board 100 includes two types of curve connection enclosing patterns 700 and 710, and a position information curve connection enclosing pattern 7 having position information.
20 are formed.

By the same procedure as in the second embodiment, the line segment and the reference position are regarded as being connected and stored, and the position information curve connection enclosing pattern 720 is used as a starting point, and the stored connection is traced to thereby enclose the curve connecting envelop. The positional relationship of the patterns is examined, and the world coordinates of the reference position are determined in order from the known world coordinates of the curve connection surrounding pattern.

According to the third embodiment, the screen coordinates and the world coordinates can be associated with each other even if the image of the calibration board is distorted and the reference position is not on a straight line. Further, the surrounding pattern can be made small, the number of reference positions can be increased in the same area, and the calibration board can be made compact.

FIG. 11 is a diagram showing a case where the camera imaging range is changed using the movement reference position 800. When the camera imaging range is, for example, the camera imaging range 810, the position information curve connection surrounding pattern 720 at the left corner is within the camera imaging range, and camera calibration is possible. However, when the camera imaging range is in the camera imaging range 820, the position information curve connection surrounding pattern 720 is outside the camera imaging range.

Therefore, the position information curve connection surrounding pattern 7
The reference position indicating the position information of 20 is set as a movable reference position 800 which is freely movable, and the movement reference position 800 is moved into the camera imaging range 820, and the position information surrounded by a square dotted line in the camera imaging range 820 in the figure. Get the curve connected enclosing pattern. Based on this, the world coordinates are changed to calibrate the camera. In this way, by making the movement reference position 800 movable like a sticker, the position information curve connection / surrounding pattern can be contained within the camera imaging range.

According to the third embodiment, the image pickup range of the camera can be changed to an arbitrary position within the calibration board.

The present invention can also be achieved by supplying a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus.
In this case, the program code itself constitutes the present invention, and the storage medium storing the program also constitutes the present invention.

[0050]

As described above, according to the present invention, it is possible to provide a camera device, a camera calibration system and a camera calibration method capable of automating the camera calibration work.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a camera calibration system according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a camera calibration device according to the first embodiment of the present invention.

FIG. 3 is a diagram showing a calibration board according to the first embodiment of the present invention.

FIG. 4 is a diagram showing a surface pattern of a calibration board according to the first embodiment of the present invention.

FIG. 5 is a flowchart showing a camera calibration processing procedure according to the first embodiment of the present invention.

FIG. 6 is a diagram showing origin identification processing and face division processing according to the first embodiment of the present invention.

FIG. 7 is a diagram showing a correspondence relationship between screen coordinates and world coordinates in the first embodiment of the present invention.

FIG. 8 is a diagram showing a linear connection surrounding pattern according to the second embodiment of the present invention.

FIG. 9 is a diagram showing a processing procedure of a linear connection surrounding pattern according to the second embodiment of the present invention.

FIG. 10 is a diagram showing a curved connection surrounding pattern according to the third embodiment of the present invention.

FIG. 11 is a diagram showing a case where the camera imaging range is changed using the movement reference position in the third embodiment of the invention.

FIG. 12 is a diagram showing a calibration board used in a conventional camera calibration system.

[Explanation of symbols]

100 calibration board 101 Reference position pattern (reference position) 102 Siege pattern 103 central pattern 104 square pattern 200 cameras 300 camera calibration device

Continued front page    F term (reference) 2F065 AA01 DD00 DD06 FF04 JJ03                       JJ19 JJ26 QQ31                 5B047 AA11 CB09 CB23                 5B057 AA01 CA12 CA16 DA07 DB02                       DC05                 5L096 BA02 CA02 FA69 MA03

Claims (11)

[Claims] 1. A camera device for performing calibration according to a calibration parameter, wherein a calibration imaging object having a reference position indicating a reference position for calibration and identification information for identifying the reference position is predetermined. A camera device, comprising: an image pickup means for picking up an image at the position; and a calibration parameter calculation means for calculating the calibration parameter based on the image of the calibration image pickup object. 2. The camera device according to claim 1, wherein the calibration imaging target object is provided with a plurality of enclosing patterns that group a plurality of the reference positions. 3. The camera device according to claim 2, wherein among the plurality of surrounding patterns, a position information surrounding pattern indicating position information in world coordinates is included as the identification information. 4. The camera device according to claim 3, wherein the calibration imaging object is provided with a plurality of surfaces on which the plurality of surrounding patterns are aligned in parallel. 5. The plurality of surrounding patterns include, as the identification information, a plurality of connecting surrounding patterns in which a positional relationship between the surrounding patterns is defined. Among the plurality of connecting surrounding patterns, 3. A position information surrounding pattern indicating position information in world coordinates according to the number of the reference positions and the arrangement of the reference positions is included.
The camera device according to 1. 6. The camera device according to claim 5, wherein the connection surrounding pattern is a linear connection surrounding pattern in which a pattern defining a positional relationship between the surrounding patterns is formed by a straight line. 7. The camera device according to claim 5, wherein the connection surrounding pattern is a curved connection surrounding pattern in which a pattern that defines a positional relationship between the surrounding patterns is formed by a curved line. 8. A reference position for changing a position information curve connection surrounding pattern, which is included as one of the curve connection surrounding patterns and indicates position information in world coordinates according to the number of the reference positions and the arrangement of the reference positions. The camera device according to claim 7, wherein is provided in the position information curve connection surrounding pattern. 9. The calibration parameter calculation means calculates reference position coordinate calculation means for calculating screen coordinates of the center of gravity of the reference position pattern for each of the surrounding patterns, and position information from the reference position in the position information surrounding pattern. Origin identifying means for extracting the reference position, surface dividing means for classifying the reference position for each surface, reference position aligning means for aligning the reference positions classified for each surface and the surrounding pattern in the row direction and the column direction, the alignment Based on the determined reference position, coordinate correspondence means for associating screen coordinates with world coordinates, and calculation means for calculating the calibration parameters based on the correspondence relationship between the screen coordinates and world coordinates. The camera device according to claim 4, wherein the camera device is a camera device. 10. A calibration imaging object in which a reference position indicating a calibration reference position and identification information for identifying the reference position are formed, and the calibration imaging object is imaged from a predetermined position. Camera calibration including a camera and a camera calibration device that calculates calibration parameters of the camera based on the captured image of the calibration imaging object and calibrates the camera according to the calculated calibration parameters system. 11. A step of capturing an image of a calibration image-capturing object from a predetermined position with a camera, in which a reference position indicating a reference position for calibration and identification information for specifying the reference position are formed, A camera calibration method comprising: a step of calculating a calibration parameter of the camera based on the image of the calibration imaging object; and a step of calibrating the camera according to the calculated calibration parameter.