JP2005182833A - Method and instrument for measuring photographic parameter and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and an instrument for exactly measuring unknown photographic parameters (distortion coefficient, focal distance) from a picture photographed by a camera whose photographic parameters are unknown. <P>SOLUTION: The measurement method includes a step (S1) where a picture as a measurement object is inputted, a step (S2) where photographic parameters to be measured are designated, a step (S3) where a picture area required for measurement of the photographic parameters in the picture as the measurement object is designated, a step (S4) where information related to the photographic parameters to be measured are extracted from the designated picture area, a step (S5) where the photographic parameters are calculated in accordance with a prescribed arithmetic operation processing based on the quantity and positions of extracted features and a step (S6) where the calculated photographic parameters are outputted. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging parameter measurement method and apparatus for measuring an unknown imaging parameter of a camera at the time of imaging based on an image captured by a certain camera, in particular, its distortion coefficient and focal length, and The present invention relates to a recording medium on which a computer program for carrying out the method is recorded.

  Conventionally, parameters at the time of photographing such as a distortion coefficient and a focal length have been measured by photographing a point where coordinate values arranged in a three-dimensional space are known and observing the photographed image. FIG. 7 is a schematic diagram showing an apparatus configuration for performing such a conventional measurement method. In FIG. 7, V is a lattice point object, and a plurality of lattice points whose two-dimensional coordinates are known are attached to the lattice point object V. Reference numeral 51 denotes a camera, which captures the lattice point object V a plurality of times while changing the photographing distance, and outputs the image data to the image display device 52 and the lattice point position extraction unit 53. The image display device 52 displays an image of the lattice point object V according to the input image data.

The grid point position extraction unit 53 extracts the position on the image of the grid point designated by the user input from the pointing device 54, obtains the image coordinates of the designated grid point, and outputs it to the imaging parameter calculation unit 55 To do. The imaging parameter calculation unit 55 is configured to calculate the imaging parameters of the camera 51 including distortion parameters and focal length parameters based on the input image coordinates of the grid points, the distance to the camera 51, and the actual three-dimensional coordinates of the grid points that are known. Is calculated and output to the imaging parameter output unit 56. The shooting parameter output unit 56 converts the calculated shooting parameter into character data or the like and outputs it to the outside.
Japanese Patent Laid-Open No. 7-15647

  Incidentally, in recent years, with the development of multimedia, processing for handling images taken with a camera has increased. In this case, image processing such as separation, composition, and editing is performed in consideration of the shooting parameters of the camera when the image is shot. However, it is unclear what kind of camera the image to be processed was shot with, and under what shooting conditions the image was obtained, that is, the shooting parameters of the camera at the time of shooting. In many cases, an image with unknown is included. In such a case, it is impossible to re-shoot using a camera whose shooting parameters are unknown. Therefore, it is impossible to obtain the shooting parameters of the camera by the conventional method as described above. As a result, there is a problem that image processing must be performed using an image in a state where the shooting parameters of the camera are not known, and accurate image processing cannot be performed.

  The present invention has been made in view of such circumstances, and accurately measures the shooting parameters (distortion coefficient, focal length) of the camera when the image is shot from the image shot by the camera whose shooting parameter is unknown. An object of the present invention is to provide an imaging parameter measurement method and apparatus capable of performing the above.

  Another object of the present invention is to provide a recording medium on which a computer program for performing measurement of such imaging parameters is recorded.

  An imaging parameter measurement method according to claim 1 is a method of measuring a focal length when an image is captured based on a captured image, the step of selecting a part of an image area from the image, and a selection A step of obtaining a feature related to a focal length to be measured from the image region, and a step of calculating a focal length according to a predetermined calculation process based on the obtained feature, and the step of selecting the image region includes: Originally, the method includes a step of selecting four image regions including four straight lines constituting a rectangle, and the step of obtaining the feature includes a step of extracting a plurality of edges from each of the selected image regions, and extraction A step of obtaining coordinates of the edge on the image, a step of obtaining a straight line for each image area based on the obtained coordinate value of the edge, and a rectangular shape having the obtained four straight lines as sides. Calculating the focal distance, calculating the focal distance based on the determined coordinates of the four vertices, and edge strength representing the sharpness of each edge. And a step of applying a large weighting when the edge is clear and a small weighting when the edge is not clear.

According to a second aspect of the present invention, there is provided a photographing parameter measurement method according to the first aspect, wherein the coordinates of the four vertices on the obtained image are (x _i , y _i ) (i = 0,..., 3), The focal length f is calculated by the following equation (1), where f ′ is the focal length of f ′ and f is the focal length to be measured.

However, in _{(1), m 0x, m 0y, m} 0f is a vector element of the _{m 0 (m 0 = (m} 0x, m 0y, m 0f) T), m 1x, m 1y, m 1f is m ₁ vector element _{(m 1 = (m 1x,} m 1y, m 1f) T) it is.
Further, m ₀ and m ₁ are represented by the following equations (2) and (3), respectively.

Further, in the equations (2) and (3), n ₀ , n ₁ , n ₂ , and n ₃ are represented by the following equations (4) to (7), respectively.

  An imaging parameter measurement apparatus according to claim 3 is an apparatus for measuring a focal length when an image is captured based on the captured image, and means for selecting a part of the image area from the image, and a selection Means for obtaining a feature related to the focal length to be measured from the image region, and means for calculating the focal length according to a predetermined calculation process based on the obtained feature, and the means for selecting the image region comprises: Includes means for selecting four image areas each including four straight lines constituting a rectangle, and the means for obtaining the features includes means for extracting a plurality of edges from each selected image area, Means for obtaining the coordinates of the edge on the image, means for obtaining a straight line for each image area based on the obtained coordinate value of the edge, and on the image of the rectangular vertex having the obtained four straight lines as sides. A means for obtaining the coordinates of The means for calculating the focal length includes a means for calculating the focal length based on the obtained coordinates of the four vertices, and an edge strength that is clear according to the edge strength representing the sharpness of each edge. And a means for applying a large weighting, and a small weighting if not clear.

According to a fourth aspect of the present invention, there is provided the imaging parameter measuring apparatus according to the third aspect, wherein the coordinates of the four vertices on the obtained image are (x _i , y _i ) (i = 0,..., 3), and an arbitrary value is assumed. The focal length f is calculated by the following equation (4), where f ′ is the focal length of f ′ and f is the focal length to be measured.

However, in _{(1), m 0x, m 0y, m} 0f is a vector element of the _{m 0 (m 0 = (m} 0x, m 0y, m 0f) T), m 1x, m 1y, m 1f is m ₁ vector element _{(m 1 = (m 1x,} m 1y, m 1f) T) it is.
Further, m ₀ and m ₁ are represented by the following equations (2) and (3), respectively.

Further, in the equations (2) and (3), n ₀ , n ₁ , n ₂ , and n ₃ are represented by the following equations (4) to (7), respectively.

  A recording medium according to claim 5 is a recording medium that records a computer program for measuring a focal length when a captured image is captured based on a captured image captured by a computer. Extracting a part of the image area from the image according to an external input; acquiring a feature related to the focal length to be measured from the extracted image area; and determining a focal distance according to a predetermined calculation process based on the acquired feature. The step of extracting the image region includes the step of extracting four image regions including four straight lines that originally formed a rectangle, and acquiring the feature. The steps include extracting a plurality of edges from each selected image region, obtaining coordinates of the extracted edges on the image, Calculating the focal length, including a step of obtaining a straight line for each image area based on the coordinate value of the image and a step of obtaining coordinates on the image of a rectangular vertex having the obtained four straight lines as sides. The step of calculating the focal length based on the obtained coordinates of the four vertices, and depending on the edge strength representing the sharpness of each edge, if the edge is clear, the weight is large, and if the edge is not clear Recorded a computer program including a step of applying a small weight.

  FIG. 1 is a configuration diagram of an imaging parameter measuring apparatus according to the present invention. The image calibration device includes a GUI that supplies an image supply device 1 that supplies a target image for measuring shooting parameters, a type of shooting parameters to be measured, and a partial image area of the image supplied from the image supply device 1. A pointing device 2 designated by (Graphical User Interface), a feature extracting unit 3 for extracting a feature for measuring a shooting parameter from an image region designated by the pointing device 2, a feature amount in the extracted feature, and A shooting parameter calculation unit 4 that calculates shooting parameters based on the feature positions;

  In addition to the above configuration, the imaging parameter measurement device of the present invention includes an imaging parameter output unit 5 that outputs the imaging parameters calculated by the imaging parameter calculation unit 4, an image supplied from the image supply device 1, And an image display device 6 for displaying the calculated photographing parameters. The imaging parameter output unit 5 outputs the imaging parameters calculated by the imaging parameter calculation unit 4 to the image display device 6 as graphic data or character patterns, and the image display device 6 displays an image to be measured. The calculated shooting parameters are displayed as text or graphics.

  Note that a character input device 7 may be provided, and the type of imaging parameter to be measured may be designated by GUI, characters, numbers, etc., instead of the pointing device 2.

  Next, an operation for measuring imaging parameters will be described. FIG. 2 is a flowchart showing the operation procedure. A target image whose shooting parameters are to be measured is input from the image supply device 1 to the feature extraction unit 3 and the image display device 6 (step S1). In the image display device 6, the target image is displayed. In addition, the type of imaging parameter to be measured (distortion coefficient, focal length) is input to the feature extraction unit 3 by the pointing device 2 or the character input device 7 (step S2). Next, the pointing device 2 designates a part of the image area for referring to the shooting parameter from the target image input to the feature extraction unit 3 (step S3). Here, an image area related to the imaging parameter is designated. For example, when measuring the focal length, an image region including a straight line that should form a rectangle in the imaging space is designated.

  Next, the feature extraction unit 3 extracts features (for example, edges) in the designated image region, and outputs the feature amounts and feature positions to the imaging parameter calculation unit 4 (step S4). The shooting parameter calculation unit 4 calculates shooting parameters based on the input feature amount and feature position, and outputs the shooting parameters to the shooting parameter output unit 5 (step S5). The calculated shooting parameters are output as graphic data or character patterns from the shooting parameter output unit 5 to the image display device 6 (step S6). The calculated shooting parameters are displayed on the image display device 6 as text or graphics.

  The feature extracted from the designated image area may be designated by the user from the outside, or may be automatically designated by image processing from the designated image area. .

  Based on images taken by cameras with unknown shooting parameters, it is possible to make assumptions of shooting conditions and measure the unknown shooting parameters, and separate them from images taken by cameras with unknown shooting parameters , Combining, editing, etc. can be performed more accurately. In addition, when measuring shooting parameters, features for calculating shooting parameters are automatically extracted from the image area specified by the user by image processing, so there is no need for the user to specify the features. The user does not need to know the quantity of the feature such as image coordinates and size.

  Hereinafter, embodiments of the present invention will be specifically described. In the following embodiment, a case where a distortion coefficient (first embodiment) and a focal length (second embodiment) are measured as imaging parameters will be described as an example.

  FIG. 3 is a schematic diagram showing a basic configuration of a system as an embodiment of the present invention. This system includes a personal computer 11 as a user terminal and an image display device 6 that displays target images and calculated shooting parameters. A display 12, a mouse 13 as a pointing device 2 connected to a personal computer 11, and a keyboard 14 as a character input device 7. The personal computer 11 loads a program for performing processing such as extraction of features in a designated image area and calculation of designated shooting parameters from a magnetic disk 15 as a recording medium on which the program is recorded. Note that the feature extraction unit 3, the shooting parameter calculation unit 4, and the shooting parameter output unit 5 in FIG. 1 are incorporated in the software of the personal computer 11.

(First embodiment: measurement of distortion coefficient)
First, a target image G1 as shown in FIG. 4 is supplied from the image supply device 1 to the feature extraction unit 3 and the display 12 (S1). The supplied image G1 is displayed on the display 12. An image G1 shown in FIG. 4 is an image of a scene in which a container (paper cup) B whose outer shape has an inverted truncated cone shape and whose upper surface is opened is placed on the desk A. As an imaging parameter to be measured, the user selects “distortion” using the mouse 13 or the keyboard 14 (S2). Next, while viewing the target image displayed on the display 12, the user uses the mouse 13 to designate a part of the image region S from the target image (S3). In this case, the designated image region S includes a portion (edge of desk A) that was originally a straight line but has become a curved line due to distortion. Instead of the image region S, a plurality of regions, for example, two image regions S1 and S2 may be designated for a curve that should have been a single straight line.

Next, the feature extraction unit 3 extracts edges from the designated image region S by an edge extraction operator such as a Sobel operator or a Canny operator, which is a method of spatial filtering (S4). When n edges are extracted from the corresponding image area S, the x-coordinate, y-coordinate on the image of each edge position, and the edge strength output by the edge extraction operator are respectively represented by x _i ′, y _i ′, E _i (i = 0,..., n−1). These obtained data are output to the shooting parameter calculation unit 4, and the shooting parameter calculation unit 4 calculates a distortion coefficient, which is a shooting parameter, according to the following method based on these data (S5). .

Assume that the image is subjected to radial distortion due to the distortion coefficient κ. That is, when the image is I ′ (x ′, y ′) (x ′ and y ′ are known), and the image when distortion is corrected is I ′ (x, y) (x and y are unknown). The following expressions (8) and (9) are established.
x ′ = x {1−κ (x ² + y ² )} (8)
y ′ = y {1−κ (x ² + y ² )} (9)

The straight line (the edge of the desk A) that should be formed by each row of edges described above is defined as the following expression (10) called the standard form of Hesse.
−x sin θ + y cos θ = ρ (10)
(However, θ and ρ are parameters that determine the straight line.
θ: Angle between the perpendicular line drawn from the origin and the x-axis
ρ: Distance from the origin of the xy coordinate system to the straight line)

  For each of the edges described above, the above-described equations (8) to (10) are obtained, and the unknown x and y are eliminated from the equations (8) to (10) to obtain three unknowns κ, θ, and ρ. A single equation expressed as follows is obtained. By performing such processing for at least three edges to obtain at least three equations represented by three unknowns, and solving these simultaneous equations, the distortion coefficient κ in the corresponding image can be obtained.

The edge strength E _i described above at each edge is used for weighting in the calculation process described above. That is, a large weight is assigned to an edge point where the edge is clear, and a small weight is assigned to an edge point where the edge is not very clear.

  It is possible to calculate the distortion coefficient κ by designating one image area and performing the above-described arithmetic processing on one straight line. However, by designating a plurality of image areas, However, if the above-described arithmetic processing is performed, the distortion coefficient κ can be comprehensively determined and the calculation accuracy can be improved. In this case, in the above-described calculation processing on each straight line, the parameters θ and ρ are different from each other on each straight line, and the parameter κ is the same on all straight lines. For example, in FIG. 4, the above-described calculation process can be performed on a straight line that should be formed by the region S and a straight line that should be formed by the regions S1 and S2.

  The distortion coefficient κ calculated as described above is sent in the form of character pattern data to the display 12 via the photographing parameter output unit 5 (S6). Then, the calculated value of the distortion coefficient κ is displayed on the display 12 together with the target image.

(Second embodiment: focal length measurement)
First, a target image G2 as shown in FIG. 5 is supplied from the image supply device 1 to the feature extraction unit 3 and the display 12 (S1). The supplied image G2 is displayed on the display 12. An image G2 illustrated in FIG. 5 is an image obtained by photographing a scene of the apartment house C. As an imaging parameter to be measured, the user selects “focal length” using the mouse 13 or the keyboard 14 (S2). Next, while viewing the target image displayed on the display 12, the user uses the mouse 13 to display four image regions R _i (i = i = 4) including four straight lines that should form a rectangle in the imaging space. 0,..., 3) are designated (S3). Note that the image region R4 may be added and designated as a portion forming a lower straight line.

Then extracted from the specified four respective image areas R _i, by the feature extraction section 3, Sobel operator, an edge by the edge extraction operator, such as Canny operator (S4). When n _i edges are extracted from each corresponding image region R _i , the x-coordinate, y-coordinate on the image at each edge position, and the edge strength output by the edge extraction operator are respectively represented as X _ij , Y _ij , Let E _ij (i = 0,..., 3, j = 0,..., N _i −1).

To the extracted edge, fit a straight line by the least squares method separately for each image region R _i. At that time, the edge strength is used for weighting. That is, a large weight is applied to an edge point where the edge is clear, and a small weight is applied to an edge point where the edge is not very clear. The obtained four straight lines are set as four sides forming a rectangle, and a rectangle whose vertex is the intersection of the four sides is obtained. The x and y coordinates on the image of the four intersections are assumed to be x _i and y _i (i = 0,..., 3), respectively.

  These obtained data are output to the shooting parameter calculation unit 4, and the shooting parameter calculation unit 4 calculates the focal length, which is the shooting parameter, according to the calculation process shown below based on these data (S5). ).

  If the temporary focal length is f ′ (arbitrary value), the desired focal length f can be obtained by the following equation (1).

However, in _{(1), m 0x, m 0y, m} 0f is a vector element of the m _{0, i.e., m 0 = (m 0x,} m 0y, m 0f) T, also m _1x, m _1y, m _1f is a vector element of m _{1, i.e., m 1 = (m 1x,} m 1y, m 1f) is ^T.
Further, m ₀ and m ₁ are represented by the following equations (2) and (3), respectively.

However, in the equations (2) and (3), n ₀ , n ₁ , n ₂ , and n ₃ are represented by the following equations (4) to (7), respectively.

  It is possible to calculate the focal length f by performing the above calculation process assuming one rectangle. However, the above calculation process is performed for each rectangle assuming a plurality of rectangles. For example, the focal length f can be comprehensively determined, and the calculation accuracy can be improved. More specifically, the image regions R5, R6, R7, and R8 as shown in FIG. 5 are further specified, and the above-described rectangle formed by four straight lines corresponding to R3, R5, R6, R7, and R8 is also described above. The arithmetic processing may be performed.

  The focal length f calculated as described above is sent in the form of character pattern data to the display 12 via the photographing parameter output unit 5 (S6). Then, the calculated value of the focal length f is displayed on the display 12 together with the target image.

  In the above-described example, the program for performing the processing of the present invention is loaded from the magnetic disk 15, but this is an example, and the recording medium for recording the program is not limited to this. FIG. 6 is a diagram illustrating another example of the recording medium. Examples of the recording medium described in the claims of the present invention include portable recording media such as the magnetic disk 15 and CD-ROM 16 described above, and personal computer 11 that can perform program communication wirelessly or by wire. May be any one of the line memory 17 provided in the memory, the RAM 18 provided in the personal computer 11, and the processing device side memory 18 such as a hard disk.

It is a block diagram of the imaging | photography parameter measurement apparatus which concerns on this invention. 5 is a flowchart showing an operation procedure of the imaging parameter measurement method according to the present invention. It is a schematic diagram which shows the system configuration | structure of embodiment of this invention. It is a figure which shows the target image and designated image area | region in 1st Embodiment. It is a figure which shows the object image and designated image area | region in 2nd Embodiment. It is a figure which shows the example of a recording medium. It is a schematic diagram which shows the apparatus structure which implements the measuring method of the conventional imaging | photography parameter.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image supply apparatus 2 Pointing apparatus 3 Feature extraction part 4 Shooting parameter calculation part 5 Shooting parameter output part 6 Image display apparatus 7 Character input device
11 Personal computer
12 display
13 mouse
14 Keyboard
15 Magnetic disk
16 CD-ROM
17 Line memory
18 Processor side memory

Claims (5)

A method for measuring a focal length when an image is captured based on a captured image, the method comprising: selecting a part of the image area from the image, and a focal distance to be measured from the selected image area And calculating a focal length according to a predetermined calculation process based on the calculated characteristics,
The step of selecting the image region includes a step of selecting four image regions including four straight lines that originally formed a rectangle, and the step of obtaining the feature includes a plurality of steps from each of the selected image regions. Extracting the edges of the image, obtaining the coordinates of the extracted edges on the image, obtaining a straight line for each image region based on the obtained coordinate values of the edges, and obtaining the obtained four straight lines Determining the coordinates of the rectangular vertices on the image on the image, and calculating the focal distance includes calculating the focal distance based on the determined coordinates of the four vertices, and sharpening each edge. And a step of applying a large weighting when the edge is clear and a small weighting when the edge is not clear, according to the edge strength representing the brightness. . The coordinates of the four vertices on the obtained image are (x _i , y _i ) (i = 0,..., 3), the temporary focal length which is an arbitrary value is f ′, and the focal length to be measured is f. In this case, the focal length f is calculated by the following equation (1).

However, in _{(1), m 0x, m 0y, m} 0f is a vector element of the _{m 0 (m 0 = (m} 0x, m 0y, m 0f) T), m 1x, m 1y, m 1f is m ₁ vector element _{(m 1 = (m 1x,} m 1y, m 1f) T) it is.
Further, m ₀ and m ₁ are represented by the following equations (2) and (3), respectively.

Further, in the equations (2) and (3), n ₀ , n ₁ , n ₂ , and n ₃ are represented by the following equations (4) to (7), respectively.

An apparatus for measuring a focal distance when an image is captured based on a captured image, and means for selecting a part of the image area from the image and a focal distance to be measured from the selected image area And a means for calculating the focal length according to a predetermined calculation process based on the calculated characteristics.
The means for selecting the image area includes means for selecting four image areas including four straight lines originally constituting a rectangle, and the means for obtaining the feature includes a plurality of means for selecting each of the selected image areas. Means for extracting the edge of the image, means for obtaining the coordinates of the extracted edge on the image, means for obtaining a straight line for each image area based on the coordinate value of the obtained edge, and each of the obtained four straight lines. Means for calculating the coordinates of the rectangular vertices on the image on the image, and the means for calculating the focal length includes means for calculating the focal length based on the obtained coordinates of the four vertices, and the sharpness of each edge. And a means for applying a large weighting when the edge is clear and a small weighting when the edge is not clear according to the edge strength representing the brightness. The coordinates of the four vertices on the obtained image are (x _i , y _i ) (i = 0,..., 3), the temporary focal length which is an arbitrary value is f ′, and the focal length to be measured is f. In this case, the focal length f is calculated by the following equation (4).

However, in _{(1), m 0x, m 0y, m} 0f is a vector element of the _{m 0 (m 0 = (m} 0x, m 0y, m 0f) T), m 1x, m 1y, m 1f is m ₁ vector element _{(m 1 = (m 1x,} m 1y, m 1f) T) it is.
Further, m ₀ and m ₁ are represented by the following equations (2) and (3), respectively.

Further, in the equations (2) and (3), n ₀ , n ₁ , n ₂ , and n ₃ are represented by the following equations (4) to (7), respectively.

A recording medium recording a computer program for measuring a focal length when a captured image is captured based on a captured image captured by a computer, and a part of the image according to an external input from the captured captured image Extracting a region, obtaining a feature related to a focal length to be measured from the extracted image region, and calculating a focal length according to a predetermined calculation process based on the obtained feature,
The step of extracting the image region includes the step of extracting four image regions including four straight lines that originally formed a rectangle, and the step of acquiring the feature comprises: A step of extracting a plurality of edges, a step of obtaining coordinates on the image of the extracted edges, a step of obtaining a straight line for each image region based on the obtained coordinate values of the edges, and the obtained four straight lines Calculating a coordinate on the image of a rectangular vertex as each side, and calculating the focal length includes calculating the focal length based on the calculated coordinates of the four vertices; According to the edge strength representing the sharpness, a computer program is recorded which includes a step of applying a large weighting when the edge is sharp and a small weighting when the edge is not sharp. A computer-readable recording medium characterized.

Images