JP2005168054A - Image pickup device, and apparatus for utilizing imaged data thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a versatile method for correcting the distortion of a photographed image. <P>SOLUTION: Points pi' and pi" and points pj' and pj" are observation points on an image plane where the same point is photographed in different camera azimuths from the same spot. Positions pio' and pio" and positions pjo' and pjo" individually represent true positions where the distortion of the observation points is corrected. A symbol C indicates the optical center of a lens system. The second and fourth order distorted aberration coefficients A, B of a distorted aberration function f(Φ)=1-AΦ<SP>2</SP>+BΦ<SP>4</SP>are estimated to minimize an error evaluation value E=ä∠pjo'Cpio'-∠pjo"Cpio"}<SP>2</SP>. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

The present invention relates to an imaging apparatus such as a digital camera that captures an image and stores the image data in a storage medium, and an apparatus that reads and processes the image data in the storage medium.

An image taken by an imaging device such as a digital camera or a video camera is stored as image data in a storage medium such as a removable memory card, and the image data stored in the storage medium is read by a desired device at a later date. , Correction processing and the like are performed and output.

  An image taken by an imaging device such as a digital camera or a video camera includes geometric distortion caused by forming an image at a position deviated from a position where the image should originally be due to distortion of a so-called lens system. Yes. Until now, efforts have been made to avoid distortion by lens design techniques. However, with lens systems such as digital cameras and video cameras, it is difficult to avoid distortion with lens design technology alone, as advanced functions such as zoom and autofocus functions become common. .

  On the other hand, with the increase in the definition of digital images, new needs such as image measurement, image composition, image editing, etc. have appeared and will continue to increase. The demand is expected to increase. However, it is difficult to say that current digital cameras and the like sufficiently satisfy the conditions for achieving such an object.

The main object of the present invention is to make it possible to easily perform high-precision correction of the brightness and hue of each photographed image with a device other than the digital camera used for photographing. Another object is to provide an imaging apparatus such as a digital camera.

According to the present invention, in an imaging device that captures an image and stores the image data in a storage medium, for each captured image data, image data and a parameter relating to image quality adjustment used at the time of capturing the image data are stored as one data. It memorize | stores in a storage medium. Specifically, the parameters are a γ conversion table value read at the time of photographing, a gain value of the image signal amplifier, an average value for each color of the image signal, a dynamic range, a distance between the lens center and the imaging surface, and the like.

Furthermore, the image pickup apparatus of the present invention is characterized in that the parameters unique to the image pickup apparatus such as the image pitch and the lens focal length are added and stored as one data in the storage medium.

In the imaging apparatus of the present invention, the parameter is stored in the first line of image data stored in a storage medium.

The present invention also provides an apparatus for reading and processing image data stored in a storage medium. The image data is read from the storage medium in which the image data and the parameters are stored as one data by the imaging device. And the parameter are separated, and the correction processing of the image data is performed using the parameter.

According to the present invention, not only image data captured by an imaging apparatus such as a digital camera, but also parameters relating to image quality adjustment used at the time of capturing the image data, and parameters specific to the imaging apparatus are integrated with the image data. Since the data is stored in the storage medium, the correction processing of the image data can be easily performed by reading the data in the storage medium with a desired device.

  Hereinafter, in order to clarify embodiments of the present invention, the present invention will be specifically described with reference to the drawings.

  First, image distortion correction will be described. FIG. 1 shows an angle Φi, Φj formed by an optical axis and incident light when a certain object is photographed with a camera, and angles Φi ′, Φj ′ with respect to the optical axis of the image, and the same object is photographed by shifting the camera angle. The relationship between the angles Φi ″ and Φj ″ (broken lines in the figure) formed by the image and the optical axis is shown. R is the distance from the optical center of the lens system to the image plane. At this time, the following expressions (1) to (5) are satisfied. However, Φ is a true value, and Φio ′, Φjo ′, Φio ″, Φjo ″ are predicted values of true values calculated from the observed values Φi ′, Φj ′, Φi ″, Φj ″.

The above equation (3) is a distortion aberration function. Here, from the information of two (or more) images obtained by photographing the same object from the same point under different photographing conditions, the second and fourth order equations (3) are used as parameters for correcting the distortion. By estimating the distortion aberration coefficients A and B, correction of image distortion due to distortion is achieved . In the above equation (5), R is the distance from the optical center of the lens system to the image plane. The value of R is assumed in advance in the “one-dimensional case” described later, and distortion aberration coefficients A and B (The value of R is often predictable to some extent). However, in the “two-dimensional case” described later, the value of R is also determined based on image information in the process of obtaining the distortion aberration coefficients A and B. However, it is natural that the R value may be similarly estimated in the “one-dimensional case”.

  Next, a specific example of processing for determining distortion correction parameters will be described. In this process, two or more sets of corresponding points (observation points) are extracted from two images obtained by photographing the same object (pattern) from one point under different conditions. For example, a common (identical) pattern is extracted from two images taken before and after the same object while shaking the camera left and right, and a specific point on the pattern is used as an observation point. As such an image, a two-frame image of a video film obtained by photographing a person or the like with a digital camera or a video camera can be cited as an example. The case where the observation points are aligned on a straight line passing through the optical axis is referred to as a “one-dimensional case”; otherwise, the observation points are distributed on a two-dimensional image plane (FIG. 2). Will be referred to as the “two-dimensional case”.

[One-dimensional example]
First, an example of distortion correction parameter determination processing in the one-dimensional case will be described. FIG. 3 is a flowchart showing the outline.

  In the first step S1, A = Ao and B = 0 are initialized. R also gives a predetermined value.

  In the next step S2, an evaluation pattern common to the two images is extracted from the two images. Then, two pairs of specific points on the evaluation pattern in one image and corresponding specific points on the evaluation pattern of the other image are detected as observation points. Since a one-dimensional case is assumed here, these observation points are aligned on a straight line passing through the optical axis. The coordinate values (Xi ', Yi'), (Xi ", Yi") in the respective coordinate systems of one pair of observation points Pi ', Pi ", and the respective coordinates of the other pair of observation points Pj', Pj" The coordinate values (Xj ′, Yj ′) and (Xj ″, Yj ″) in the system are expressed by the angle variable Φ according to the above equation (5). Ie

  In step S3, Φi ′, Φi ″ and Φj ′, Φj ″ obtained in the previous step are substituted into Φ ′ in the equation (7) to obtain a solution Φ. Let the solution be Φio ′, Φjo ′, Φio ″, Φjo ″.

This equation (7) is an equation obtained by substituting the above equation (3) into the above equation (1). Here, B = 0, so this equation is a cubic irreducible equation AΦ ³ -Φ + Φ '= 0
It becomes. In order to solve this, the cubic equation (Cardano formula)
p = −1 / (3A), q = Φ ′ / (2A)
Followed by variable conversion as follows.

Thus, the solution is expressed as follows, but since only real solutions are considered here, only Φ1 has to be used.

  In the case of one-dimensional observation points arranged on a straight line passing through the optical axis, the following error evaluation formula can be obtained by substituting the solution Φ1 into the error evaluation formula (4).

  When the solution corresponding to Φ ′, Φ ″ (measured value) in the equation (7) is set as Θ ′, Θ ″, Θ ′, Θ ″ is the parameter A and the measured values (X ′, Y ′), (X ″ , Y ″), E can also be expressed as a function of A, and therefore ∂E / ∂A can be obtained from equation (7). Similarly, ∂E / ∂B can also be obtained. it can.

  In step S4, the error evaluation value E is calculated by the equation (10), and the convergence of the error evaluation value E is determined. When it is determined that the error evaluation value E has not converged, in the next step S5, an increment ΔA of A is obtained from the relational expression of ΔE = (∂E / ∂A) ΔA (the above equation (10) or the following ( 12) Refer to equation, where ΔE = E), the value obtained by adding the value of ΔA to the current value of A is set as the value of A, and the process returns to step S3. The processing loop from step S3 to step S5 is repeated until it is determined in step S4 that it has converged.

  If it is determined in step S4 that the value has converged, since the value of A has been determined, the process proceeds to step S6 to determine B, and B is set to the initial value Bo. In the next step S7, it is checked whether there is another common pattern for determining B from the two images. If another common pattern is not found, the process ends (B is determined by the initial value Bo).

  If another common pattern is found, in the next step S8, as in step S2, the common pattern is used as an evaluation pattern, and two pairs of corresponding observation points on each evaluation pattern are detected. The coordinates of each observation point are represented by an angle variable Φ.

  In the next step S9, Φi ′, Φi ″, Φj ′, Φj ″ obtained in the previous step are substituted into the equation (7) to obtain a solution Φ. Let the solutions be Φ′io, Φ′jo, Φio ″, Φjo ″. In the next step S10, the error evaluation value E is calculated by the above equation (10), and the convergence is determined. If it is determined that the error evaluation value E has not converged, in the next step S11, an increment ΔB of B is obtained from the relational expression of ΔE = (∂E / ∂B) ΔB (the above equation (10) or a later description). Refer to equation (11) or equation (12), where ΔE = E). Then, a value obtained by adding the value of ΔB obtained in the previous step to the current value of B is set as the value of B, and the process returns to step S9. The processing loop from step S9 to step S11 is repeated until it is determined in step S10 that the error evaluation value E has converged.

  If it is determined in step S10 that E has converged, both A and B have been determined, and the process ends. As described above, A and B are determined. Since R is also known in advance, the necessary distortion correction parameters were determined.

[Example in the case of two dimensions]
Next, an example of distortion correction parameter determination processing in the two-dimensional case will be described. FIG. 4 is a flowchart showing the outline.

  In the first step S21, initial setting of A = Ao, R = Ro, and B = 0 is performed, and an appropriately small value is set as ΔR.

  In the next step S22, a pattern common to the two images is extracted as an evaluation pattern. Then, two pairs of specific points on the evaluation pattern in one image and corresponding specific points on the evaluation pattern of the other image are detected as observation points. Coordinate values (Xi ', Yi'), (Xi ", Yi") in the coordinate system of one pair of observation points Pi ', Pi ", and the coordinate systems of the other pair of observation points Pj', Pj" The coordinate values (Xj ′, Yj ′), (Xj ″, Yj ″) at are expressed by the angle variable Φ by the above equation (6).

  Here, unlike the above-described one-dimensional case, the relative positional relationship between the evaluation pattern or the observation points is arbitrary. Such an observation point and distortion correction are schematically illustrated in FIG. In FIG. 2, pi 'and pi "(or pj' and pj") are observation points on the image plane obtained by photographing the same point with different camera orientations from the same point, and this is a set of detection points detected in step S22. A pair of observation points, pio 'and pio "(or pjo' and pjo") represent the true position where the distortion of the observation point is corrected. In FIG. 2, pi 'and pi' are originally in the same direction, but are intentionally shifted. C represents the optical center of the lens system.

Now, since the angle between objects does not depend on the change in camera orientation (even if there is a rotation around the optical axis), 視点 pjo'Cpio '= ∠pjo "Cpio"
The above relationship always holds (constraint condition 1). That is, the following equation holds.

  The expression (11) is an expression for evaluating an error by an angle. If this is expressed by an (x, y) coordinate value, the following error evaluation expression can be obtained.

  Return to the description of the process. In step S23, Φi ′, Φi ″, Φj ′, Φj ″ obtained in the previous step is substituted into Φ ′ in the equation (7) to obtain a solution Φ. Let the solutions be Φio ′, Φjo ′, Φio ″, Φjo ″. In the next step S24, the error evaluation value E is calculated by the equation (11), and the convergence of the error evaluation value E is determined. When it is determined that it has converged, the process proceeds to step S26. Otherwise, the process proceeds to step S25, and ΔA is obtained from the relational expression of ΔE = (∂E / ∂A) ΔA (refer to the equation (10), (11) or (12), where ΔE = E). . Then, the value obtained by adding the value of ΔA to the current value of A is set as the value of A, and the process returns to step S23. Until it is determined that the error evaluation value E has converged, the processing loop from step S23 to step S25 is repeated.

  If it is determined in step S24 that E has converged, the process proceeds to step S26. In step S26, it is checked whether another pattern common to the two images is found. If not found, the process ends. If found, the common pattern is used as a new evaluation pattern, and the value of A (represented as A ') is obtained again by the processing from step S27.

  First, in step S27, an initial value A'o is set to A '. In the next step S28, as in step S22, two pairs of corresponding observation points on the new evaluation pattern of the two images are detected, and the coordinate value of the observation point is converted into the angle variable Φ.

  In the next step S29, the solution Φ is obtained by substituting Φi ′, Φi ″, Φj ′, Φj ″ obtained in the previous step into Φ ′ in the equation (7). Let the solutions be Φio ′, Φjo ′, Φio ″, Φjo ″. In the next step S30, the error evaluation value E is calculated by the equation (11) (or the equation (12)), and the convergence of the error evaluation value E is determined. If it is determined that the process has converged, the process proceeds to step S32. If not, the process proceeds to step S31, and ΔA ′ is obtained from the relationship ΔE = (∂E / ∂A ′) ΔA ′, and this value is obtained as the current value of A ′. The value added to is set as the value of A ′ and the process returns to step S29.

  When the error evaluation value E is converged by the processing loop from step S29 to step S31, the process exits from the loop and proceeds to step S32 to determine convergence of | A−A ′ |. If it is determined that the value has not converged, a value obtained by adding the value of ΔR to the current value of R in step 33 is set as a new value of R, and an initial value Ao is set in A, and then step S22 and subsequent steps. The above process is executed again. That is, R is changed and the value of A is obtained again. The evaluation pattern and the observation point used in the second and subsequent steps S22 may be the same as those used in the first step 22, and the necessary processing uses the new R value to obtain the same observation point coordinate values as in the first time. It is only the processing to re-express with the angle variable. This is the same when the process proceeds to step S28 after that, and it is only necessary to re-express the coordinate value of the same observation point detected in the first step 28 as an angle variable using the new R value.

  Now, if it is determined in step S32 that convergence has occurred, the current value of A or A ′ is determined as the value of A, and the value of R is also determined. Next, in order to determine the value of B Proceed to step S34.

  In step S34, an initial value Bo is set to B. In the next step S35, the solution Φ is obtained by substituting Φi ′, Φi ″, Φj ′, Φj ″ obtained last in step S22 or step S28 into Φ ′ in the equation (7). In the next step S36, the error evaluation value E is calculated by the equation (11) (or equation (12)), and the convergence of the error evaluation value E is determined. If it is determined that the values have converged, the values of the distortion correction parameters A, B, and R are all determined, and the process ends. If it is determined that the value has not converged, the process proceeds to step S37, where ΔB is obtained from the relational expression ΔE = (∂E / ∂B) ΔB, and a value obtained by adding the value of ΔB to the current value of B is used as the value of B. The process returns to step S35. The processing loop from step S35 to step 37 is repeated until it is determined in step S36 that the error evaluation value E has converged.

  In general, since the distortion of the lens system is not so large, under the condition that the camera does not rotate around the optical axis (constraint condition 2), as shown in FIG. When the corresponding patterns on the image are connected to each other, this represents a movement vector of the image according to the camera angle change. This is a group of substantially parallel movement vectors. With this translation vector as S, the relationship pjo'pjo "// pio'pio" // S, that is, the following relational expression holds. S can be measured in advance.

Needless to say, when the constraint condition 2 is applicable, the calculation accuracy can be improved by using the equations (13) and (14). It is also possible to control the calculation speed and the calculation speed by appropriately selecting a constraint condition to be applied from among several constraint conditions.

  It is not necessary to determine the distortion correction parameter at one time, and it may be determined at a plurality of cycles. For example, small areas are extracted from the common pattern in the order of strong contrast (for example, in the case of a human face image, the bright part of the eyes, then the bright part of the nose, etc.) You may make it determine the parameter for distortion correction in multiple cycles using an observation point. In this case, the second-order distortion may be corrected first, and then the fourth-order distortion may be corrected.

  After determining all the parameters necessary for distortion correction, error evaluation is performed in order from the center of the two images to increase the sensitivity of error evaluation at the center of the image, and then the determined parameters are corrected. However, the error may be measured and evaluated toward the periphery of the image.

  An example of a system configuration for performing the processing described above is shown in FIG. In FIG. 6, image data to be subjected to distortion correction is stored in an image file 51, and necessary two (or more) image data are read out to two or more image buffers 52. The pattern extraction unit 53 extracts an evaluation pattern common to images on the image buffer 52. The arithmetic control unit 54 detects the observation point on the extracted evaluation pattern, and determines the distortion correction parameter by executing the processing as described above. The memory 55 is used to store a program for this processing and data related to the processing.

  If the values of the parameters A, B, and R are determined in this way, the distortion of the image can be corrected. For example, the coordinates of each pixel on the image subjected to parameter estimation are converted into an angle variable by equation (6), and the angle is substituted into Φ ′ in equation (7) to obtain the true angle Φ. Can be converted into coordinates on the corrected image by equation (5), and the process of giving the pixel value of the image before correction to the coordinates can be repeated to obtain an image in which distortion due to distortion is corrected with high accuracy. it can.

Next, a digital camera (imaging device) according to the present invention will be described. FIG. 7 shows an example of a digital camera according to the present invention. The digital camera shown here includes a camera body 100 and an optical unit 101, and the camera body 100 includes an image memory (storage medium) 102 including a removable memory card as an external memory. The optical unit 101 includes a lens system, a zoom drive mechanism, a focus adjustment mechanism, a diaphragm mechanism, and the like similar to a lens barrel unit of a general digital camera.

  The camera main body 100 is provided with a CCD 103 which is an image pickup element on which an image of a subject is formed by the optical unit 101, reading of an image signal from the CCD 103, amplification of the image signal, γ conversion, analog-digital conversion, etc. Image signal circuit system 104, drive circuit system 105 for zoom drive mechanism, focus adjustment mechanism, aperture mechanism, etc. in optical unit 101, built-in computer 106 comprising CPU, RAM, ROM, etc. for control and data processing of each part of the camera , A data memory 107 for storing various parameters, an operation keyboard 108 for operating camera functions, an interface unit 109 for exchanging information between the built-in computer 106 and the operation keyboard 108 and an external computer, and the like. Although not done, the above parts operate from the battery Consisting of a power supply circuit for supplying the power. The data memory 107 and the image memory 102 can be accessed via an interface unit 109 and a built-in computer 106 from an external computer connected to the camera.

The parameters possessed by this digital camera are illustrated in FIG. 8 together with the setting method. Here, the distance R, F value between the lens center and the imaging surface, the γ conversion table value, the gain of the image signal amplifier (image signal amplifier), the average value for each color of the image signal and the dynamic range are parameters relating to image quality adjustment. Sometimes it is read. The operation keyboard 108 is used not only for operating camera functions during shooting, but also for specifying the storage location of these parameters. In addition to the parameters shown in FIG. 8, there are various parameters related to shooting conditions such as zoom driving, focus adjustment driving, aperture driving, and the like, which are not directly related to the features of the present invention, and therefore will not be described.

Among the parameters shown in FIG. 8, A and B are second-order and fourth-order distortion coefficients as described above, R is the distance between the optical center of the lens system and the image plane, that is, the imaging plane of the CCD 103, and Px and Py. Is the pixel pitch of the CCD 103 in the horizontal and vertical directions. When the origin of the coordinate system of the captured image is the intersection of the optical axis of the lens system and the imaging surface of the CCD, the pixel coordinates (x ′, y ′) on the captured image can be obtained by using the pixel pitches Px and Py (x ', Y') = (k '* Px, l' * Py)
It can be expressed as. When the coordinates after distortion correction of this pixel are set as (x, y),
(K ′ * Px) / (l ′ * Py) = x ′ / y ′ = x / y
Holds. (K ′, l ′) is a digital coordinate of the pixel (x ′, y ′) on the captured image. In the description of the estimation processing and distortion correction processing of the parameters A and B described above, this is actually Such digital coordinates are used, but this was not mentioned for the sake of simplicity. Therefore, if A, B, Px, Py, and R are known, it is possible to correct distortion of the captured image as described above.

  Px and Py are parameters inherent to the CCD 103 and are normally unchanged. Therefore, Px and Py are set when the camera is manufactured and stored in the data memory 107 in the camera body. It can also be stored in the image memory 102 as a part. Although A and B are essentially unchanged, they cannot be accurately determined at the time of manufacture due to variations at the time of camera manufacture. Therefore, the data memory is obtained by an external computer or the camera's built-in computer 106 at the beginning of use of the camera or at any time. 107, which can also be stored in the image memory 102 as part of the image data as will be described later. R is detected by the drive circuit system 105 at the time of photographing each image and is stored in the image memory 102 as a part of the image data, but is stored in the data memory 107 in correspondence with each image. It can also be made.

  When taking an image, the dynamic range of the CCD 103 is not so large, so it is required to capture an image signal in a range that matches the human visual sensitivity. In the general allocation of 8 bits per color, the color balance is set so that the total average of the input image signals is monochrome, and the maximum value and the minimum value of the image signals are scheduled to be 8 bits. Bits are allocated by adjusting the gain so that it falls within the range, that is, the median value of the luminance distribution is at the center of the range. The γ table and the gain value of the image signal amplifier are responsible for this, and these can be adaptively changed according to the photographing conditions.

FIG. 9 is a relationship diagram between the γ conversion table (γ1, γ2), the gain value of the image signal amplifier, and the input signal I and the output signal O related to this, and the output signals O1, O2, and O3 having three different values are shown. Given that, given the difference between the γ table (γ1 and γ2) and its dynamic range (D1m_D1M and D2m_D2M), the restored input signals (I11, I12, I13 and I21, I22, I23) change. Yes. Input signal I is generally I = [image signal] × [gain value]
It is represented by

  Regarding color balance, each color signal is calculated on the assumption that “the sum of all color signals is monochrome” for each image, so if the color distribution of the subject facing the camera is biased, The color of the image shifts. This is particularly problematic when a plurality of images taken while changing the camera direction are connected. For each camera orientation, the image is bluish or reddish depending on the landscape, so the total value of the color components usually changes. The same applies to the dynamic range. The input light amount and the distribution shape of the input light change for each image. Therefore, when combining multiple images, it is necessary to rebalance the colors among these multiple images or to reallocate the image signal, but the process is performed after restoring the original image data. It is desirable to do so.

  In order to restore the original image data, it is desirable that the used γ conversion table and the gain value of the image signal amplifier are stored for each image and can be read out later for the dynamic range. As for color matching, if the image signal average value for each color and the gain value defining the dynamic range can be stored for each image and read later, more accurate signal restoration can be performed. Since the types of γ conversion tables are limited, the data is normally stored in the internal data memory 107, but can also be stored in the image memory 102 as part of the image data as will be described later. Since the gain value of the image signal amplifier, the average value for each color of the image signal, and the dynamic range usually change at every photographing, they are stored in the image memory 102 as a part of each piece of image data. It is also possible to store in the data memory 107 in association with one image.

  Next, a method for obtaining and setting the distortion aberration coefficients A and B of the image when the camera is used for the first time or at any time will be described. FIG. 10 is a flowchart showing the procedure.

  First, a first image is taken from a certain point at a camera angle at which a certain subject is at the center of the angle of view (step S50). A second image is shot from the same point with the same object slightly shaken by a camera angle so that, for example, the object is located at a position shifted by 5 degrees to the left or right from the center (step S51). Data of two captured images is stored in the image memory 102, and a value of the distance R between the lens center and the imaging surface at the time of shooting is stored in the data memory 107.

  Next, the distortion aberration coefficients A and B are estimated. This can be performed using the camera's built-in computer 106 or an external computer. Either can be selected by the operation. In the former case, the built-in computer 106 estimates A and B according to the procedure shown in the flowchart of FIG. 3 using the data of the two images and the R value (step S53), and stores the result in the data memory. (Step S54). Although it is possible to use the procedure shown in the flowchart of FIG. 4, since the value of R is known, it is assumed that the value of R at the time of shooting the first and second images does not vary greatly. In this case, it is more efficient to use the procedure of FIG. As described above, in order to execute the A and B estimation processing by the built-in computer 106, the program must be stored in the ROM or RAM of the built-in computer 106, so that there is no room in the memory capacity on the camera side. In some cases, you can choose to use an external computer.

  When an external computer is used, data transfer is instructed from the operation keyboard 108 with the external computer connected to the camera, and the two image data and R are transmitted via the interface unit 109 under the control of the built-in computer 106. Is transferred to the external computer (step S55). In the external computer, A and B are estimated using the image data and R value received from the camera by a dedicated program for executing the estimation processing procedure shown in the flowchart of FIG. 3 (or FIG. 4) (step S56). The estimation result is transferred to the camera (step S57). The built-in computer 106 of the camera takes in the transferred estimated values A and B via the interface unit 109 and stores them in the data memory 107 (step S58). It is also possible to input the estimated values A and B by operating the operation keyboard 108 and store them in the data memory 107.

  Now, during normal shooting, image data is stored in the image memory 102, but the parameter storage location illustrated in FIG. 8 can be selected by the photographer by operating the operation keyboard 108. The reason for selecting the storage location of this parameter is to make it possible to flexibly cope with the software environment used for the correction processing of the captured image.

  When an external computer is connected to the camera and image correction processing is executed using the image correction processing program on the computer, the image can be stored even if the parameter is stored in the data memory 107 if determined beforehand. It may be stored in the memory 102. For example, pixel pitches Px, Py, lens focal length f, distortion aberration coefficients A, B are stored in the data memory 107, distance R between lens center and imaging surface, F value, γ conversion table value, and gain value of image signal amplifier. The image signal average value (may be a total value) and the dynamic range of the image signal are stored in the image memory 102 as a part of each piece of image data, and the image data and parameters are stored in an image correction processing program on an external computer. It can be taken from each memory location. The same applies to the storage location when the built-in computer 106 corrects distortion aberration and the like.

  However, when there is no arrangement for the storage location of such parameters, or when it is desired to perform image correction processing with an external computer not connected to the camera, at least the parameters necessary for image correction shown in FIG. It is convenient to store it in the image memory 102 as part of the data. For example, if all parameters are stored as a part of image data in a memory card as the image memory 102, the memory card is pulled out from the camera and inserted into a memory card insertion part of an external computer not connected to the camera. Thus, the data can be read, and the data can be separated into image data and parameters, and image correction processing can be executed. Alternatively, data in the image memory 102 can be transferred to an external computer connected to the camera, and image correction processing can be performed by separating the image data and parameters in the external computer.

  In any case, parameters necessary for correcting distortion, brightness, and hue of each photographed image can be easily taken into an external computer, so that a plurality of images can be combined. Image distortion, brightness, and hue can be corrected with high accuracy.

  Note that there are standards such as SISRFIF, EXIF, and TIFF / EP as standard formats of image data, but none of them has a place for storing parameters targeted by the present invention. However, since image data has a high degree of redundancy, even if a part of the image data is replaced with a parameter, there is almost no adverse effect in practical use. Usually, image data is expressed by numerical values of 8 bits or 16 bits, and therefore, for example, parameters can be stored using the first one line of the image. The example of the digital camera according to the present invention has been described above, but the present invention can also be applied to other imaging devices such as a video camera and a scanner.

It is a figure for demonstrating the basic relational expression of distortion aberration correction. It is a figure which shows the position on the image plane of the same pattern image | photographed with different camera directions, and the position after the distortion correction | amendment. It is a flowchart which shows an example of the parameter determination process for distortion correction. It is a flowchart which shows another example of the parameter determination process for distortion correction. It is a figure explaining the movement vector of an image when there is no rotation around the optical axis of a camera. It is a block diagram which shows an example of the system configuration | structure for performing the parameter determination process for distortion correction. It is a schematic block diagram of an example of the digital camera by this invention. It is a figure which shows the example of the parameter which the digital camera by this invention has. It is a figure which shows the relationship between the gamma conversion table, the gain value of an image signal amplifier, the input signal relevant to it, and an output signal. It is a flowchart for demonstrating the procedure which calculates | requires and sets a distortion aberration coefficient.

Explanation of symbols

51 Image File 52 Image Buffer 53 Pattern Extraction Unit 54 Operation Control Unit 55 Memory 100 Camera Body 101 Optical Unit 102 Image Memory (External Memory)
103 CCD (imaging device)
104 Image signal circuit system 105 Drive circuit system 106 Built-in computer 107 Data memory (internal memory)
108 Operation keyboard 109 Interface section

Claims (9)

  A plurality of sets of corresponding observation points are detected from a plurality of images including a common pattern photographed from one point by an imaging device such as a digital camera, and the angle of these observation points with respect to the optical axis of the imaging system of the imaging device is determined. A parameter determination method for image distortion correction, characterized in that a parameter for distortion correction for the image is estimated based on the calculated angle information.   2. The image distortion correction parameter determination method according to claim 1, wherein a distortion aberration coefficient of the lens is estimated by assuming a distance between an optical center of the lens of the imaging system and the image plane. Decision method.   2. The image distortion correction parameter determination method according to claim 1, wherein the distortion aberration coefficient of the lens of the imaging system and the distance between the optical center of the lens and the image plane are estimated.   An image pickup apparatus such as a digital camera for taking an image, wherein parameters for correcting distortion of the taken image are stored in a memory accessible from outside.   An image pickup apparatus such as a digital camera for taking an image, wherein parameters for correcting distortion of the taken image are stored in a memory accessible from outside as part of image data.   An image pickup apparatus such as a digital camera for taking an image, wherein parameters for correcting distortion, brightness, and hue of the taken image are stored in a memory accessible from outside.   7. The imaging apparatus according to claim 6, wherein at least some of the parameters are stored in a memory accessible from outside as part of image data.   A photographing apparatus such as a digital camera for photographing an image has a plurality of memories, and a storage location of parameters for correcting distortion, brightness, and hue of the photographed image can be arbitrarily selected from the plurality of memories. An imaging apparatus characterized by that.   The image distortion correction parameters are estimated from the two photographed images by the method according to claim 1 and stored in a memory accessible from outside. 8. The imaging device according to 8.

Images