JP4719553B2 - Imaging apparatus, imaging method, computer program, and computer-readable storage medium - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, a computer program, and a computer-readable storage medium, and is particularly suitable for use in correcting distortion aberration of an optical lens.

In recent years, video cameras have become smaller and lighter, and optical zooms have been increased in magnification. However, such a small and lightweight video camera easily causes camera shake at the time of shooting, and the effect of camera shake increases as the zoom magnification increases.
Therefore, various shake correction means have been proposed in order to remove such camera shake during photographing and obtain a stable image.

In JP-A-5-257196, a motion vector is detected from the image signal of the previous field and the image signal of the current field, and the blur is corrected by controlling the reading position from the recording means according to the detected motion vector. A technique is disclosed.
In Japanese Patent Application Laid-Open No. 11-146260, the motion of an image is detected from an image signal, and the motion of the imaging device itself is detected by an angular velocity sensor. Then, a method is disclosed in which the detected two pieces of motion information are combined and the shake is optically corrected by controlling the variable apex angle prism based on the combined two pieces of motion information.

By the way, when a subject image is picked up by a video camera, distortion aberration is generated based on the characteristics of the attached lens.
The distortion includes a barrel distortion 101 shown in FIG. 17A and a pincushion distortion 102 shown in FIG. 17B. Generally, when the optical system has a short focus, the distortion becomes a barrel distortion 101, and the long focus. In this case, the pincushion distortion 102 is obtained.

When distortion occurs, the image shifts in the radial direction from the center of the screen as shown in FIG. 18, and the image that should be imaged at the dotted line is shifted to the solid line.
For this reason, even when the same subject is photographed, the subject image is deformed depending on the position where the image is formed on the image sensor.

  The point that the subject image is deformed in this way will be described with reference to FIGS. 18 (A), (B), and (C). FIG. 18A shows a case where the subject image 111 is formed at the center of the image sensor. As shown in FIG. 18A, the subject image 111 is hardly deformed because the influence of the distortion aberration is small in the central portion of the screen. On the other hand, as shown in FIGS. 18B and 18C, when the subject images 112 and 113 are formed at the edge of the screen, the subject image is deformed because of the great influence of distortion.

  A method for correcting such distortion is disclosed in Japanese Patent Laid-Open No. Hei 6-197261. More specifically, first, the reading position from the recording means is determined based on the motion vector detection result. Thereafter, the shape of the scanning line necessary for image correction is calculated in accordance with the distortion information of the optical system. Then, the camera shake and the distortion aberration are simultaneously corrected by controlling the read address from the recording means based on the information on the calculated shape of the scanning line.

JP-A-5-257196 JP-A-11-146260 JP-A-6-197261

  However, in the above-described configuration of the prior art, a motion vector is detected between images having distortion of the optical system. For this reason, when the subject image formed on the image sensor moves from the position of FIG. 18B to the position of FIG. 18C, block matching becomes difficult when the subject image is deformed. Thus, there is a problem that the detection accuracy of the motion vector is lowered.

As described above, the conventional technique has a problem that it is difficult to accurately detect the motion vector due to the influence of the distortion of the optical system.
The present invention has been made in view of such problems, and an object thereof is to make it possible to detect a motion vector with high accuracy.

An image pickup apparatus according to the present invention includes an image pickup element that converts an object image formed by an optical lens that forms an object on an image pickup surface into an electric signal, and distortion correction data for correcting distortion of the optical lens. Based on the distortion aberration correction data calculated by the calculation means, the distortion aberration of the optical lens is corrected with respect to the image based on the subject image converted into an electrical signal by the imaging device. A distortion correction unit; a motion vector detection unit that detects a motion vector using an image in which the distortion is corrected by the distortion correction unit; and a motion correction unit that corrects a motion based on the detection of the motion vector. And the distortion correction means corrects the distortion aberration of the optical lens only in a region necessary for detecting a motion vector .
In another example of the imaging apparatus of the present invention, an imaging element that converts an object image formed by an optical lens that forms an image of an object on an imaging surface into an electrical signal, and a distortion aberration for correcting the distortion of the optical lens. Calculating means for calculating distortion aberration correction data; and based on the distortion aberration correction data calculated by the calculating means, an image of the optical lens with respect to an image based on a subject image converted into an electrical signal by the imaging device. A distortion correction unit that corrects distortion, a motion vector detection unit that detects a motion vector using an image in which the distortion is corrected by the distortion correction unit, and a motion correction based on the detection of the motion vector. A motion correcting means; and a communication means for communicating with an interchangeable lens apparatus having a table for correcting the influence of distortion of the optical lens, and the calculation The stage includes a distortion corresponding to the focal length of the optical lens and the coordinates of the image affected by the distortion of the optical lens from the table of the interchangeable lens device via the communication unit. Aberration correction coordinates are acquired, and the distortion correction data is calculated using the acquired distortion correction coordinates.
In another example of the imaging apparatus according to the present invention, an imaging element that converts an object image formed by an optical lens that forms an image of an object on an imaging surface into an electrical signal, and correction for distortion aberration of the optical lens. Calculating means for calculating distortion aberration correction data; and based on the distortion aberration correction data calculated by the calculating means, an image of the optical lens with respect to an image based on a subject image converted into an electrical signal by the imaging device. A distortion correction unit that corrects distortion, a motion vector detection unit that detects a motion vector using an image in which the distortion is corrected by the distortion correction unit, and a motion correction based on the detection of the motion vector. A motion correcting means, and a communication means for communicating with an interchangeable lens apparatus having a table for correcting the influence of distortion of the optical lens, The output means acquires a distortion coefficient corresponding to the focal length of the optical lens from the table of the interchangeable lens device via the communication means, and corrects distortion aberration using the acquired distortion coefficient. It is characterized by calculating business data.

The imaging method of the present invention is an imaging method using an imaging device including an imaging element that converts an object image formed by an optical lens that forms an image of an object on an imaging surface into an electrical signal. An image based on a subject image converted into an electrical signal by the imaging element based on a calculation process for calculating distortion aberration correction data for correcting aberrations and the distortion aberration correction data calculated in the calculation process. On the other hand, a distortion correction step of correcting distortion aberration of the optical lens, a motion vector detection step of detecting a motion vector using an image in which the distortion aberration is corrected by the distortion correction step, and detection of the motion vector based and a motion compensation step of correcting the motion, especially to correct distortion of the distortion correction process is the optical lens only space required to detect the motion vector To.

The computer program of the present invention causes a computer to execute each step of the imaging method.
A computer-readable storage medium according to the present invention stores the computer program.

  According to the present invention, since the motion vector is detected between images in which the distortion of the optical lens is corrected, the detection accuracy of the motion vector can be improved even in an imaging device affected by the distortion by the optical lens system. Can be improved.

(First embodiment)
A first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating an example of the configuration of the imaging apparatus according to the present embodiment.
In FIG. 1, 1 and 4 are fixed lenses, 2 is a zoom lens that performs zooming, 3 is a stop, and 5 is a focus lens that performs focus adjustment. Reference numeral 6 denotes an image sensor, and 7 denotes an analog front end (AFE) that performs correlated double sampling and analog-digital conversion (A / D conversion). A distortion correction unit 8 corrects distortion. A zoom encoder 9 detects the focal length of an optical lens system including the fixed lenses 1 and 4, the zoom lens 2, the diaphragm 3, and the focus lens 5. A timing generator 10 controls the reading position of the image sensor 6.

  A distortion correction coordinate recording unit 11 records distortion correction coordinates (coordinates used for distortion correction processing) corresponding to the focal length of the optical lens system and the reading position of the image sensor 6. The calculation method of the distortion correction coordinates will be described. For example, first, a test image 201 as shown in FIG. Then, distortion correction coordinates corresponding to the focal length of the optical lens system and the reading position of the image sensor 6 are calculated from the photographed test image 201 and the output image 202 of the image pickup apparatus as shown in FIG. To do. The distortion correction coordinates calculated in this way are recorded in advance in the distortion correction coordinate recording unit 11.

FIG. 3 is a diagram illustrating an example of recorded contents recorded in the table 300 provided in the distortion correction coordinate recording unit 11. FIG. 4 is a diagram illustrating an example of the relationship between the test image 201 and the output image 202 of the imaging apparatus.
As shown in FIG. 4, the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging device are the coordinates P (x, y) on the test image 201 (ideal coordinates without distortion). The image is shifted by the influence of distortion. The influence of this distortion varies depending on the focal length of the optical lens system and the distance from the center coordinates O (0, 0) of the image shown in FIG. 4 to an arbitrary point on the image. Therefore, in the present embodiment, as shown in FIG. 3, the focal length of the optical lens system, the coordinates P (x, y) on the test image 201, and the coordinates P (x ′, y ′) in association with each other and recorded in advance in the distortion correction coordinate recording unit 11.

  By the way, the coordinates P (x, y) on the test image 201 are recorded in a table for all of the focal length in the optical lens system and the coordinates P (x ′, y ′) on the output image 202 of the imaging device. In this case, the recording capacity becomes large. Therefore, in the present embodiment, the coordinates P (x, y) on the test image 201 and the coordinates P (() on the output image 202 of the imaging device are set for each predetermined focal length (for example, every 5 mm as shown in FIG. 3). x ′, y ′) are recorded in the table 300. Further, at each focal length, three coordinates P (x, y) on the test image 201 and three coordinates P (x ′, y ′) on the output image 202 of the imaging device are recorded in the table 300. I have to. The table 300 provided in the distortion correction coordinate recording unit 11 can be realized by using a flash memory (not shown), for example.

  Reference numeral 12 denotes an image recording unit that records an image signal after distortion correction, and reference numeral 13 denotes an outline extraction unit. Reference numeral 14 denotes a previous field memory for recording the image signal of the previous field, and reference numeral 15 denotes a current field memory for recording the current image signal. Reference numeral 16 denotes a motion vector calculation unit that calculates a motion vector, and reference numeral 17 denotes an address control unit that performs screen segmentation control.

Next, an example of the operation of the imaging apparatus according to the present embodiment will be described.
As described above, as the optical lens system, the fixed lens 1, the zoom lens 2 that performs zooming, the diaphragm 3, the fixed lens 4, and the focus lens 5 that performs focus adjustment are arranged on the optical axis. The subject is imaged on the imaging surface.

The image sensor 6 photoelectrically converts a subject image formed on the imaging surface by the optical lens system to generate an image signal.
The analog front end 7 performs correlated double sampling for removing reset noise included in the output signal from the image sensor 6 and A / D conversion, generates a digitized image signal, and outputs it to the distortion correction circuit 8. To do.
The zoom encoder 9 detects the position of the zoom lens 2 and calculates the focal length of the optical lens system. The timing generator 10 controls the read position of the image sensor 6 and outputs the read position information of the image sensor 6 to the distortion correction coordinate recording unit 11.

  The distortion correction coordinate recording unit 11 inputs the focal length of the optical lens system from the zoom encoder 9. Further, the distortion correction coordinate recording unit 11 inputs the reading position of the image sensor 6 from the timing generator 10. The distortion correction coordinate recording unit 11 corresponds to the input focal length of the optical lens system and the reading position of the image sensor 6 (the coordinates P ′ (x ′, y ′) illustrated in FIGS. 3 and 4). The coordinates P (x, y) on the test image 201 are read from the table 300 as distortion correction coordinates. The read coordinates P (x, y) are output to the distortion correction unit 8.

  As described above, in this embodiment, for each predetermined focal length, the coordinates P (x, y) on the test image 201 and the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging device are Are recorded in the table 300 three by three. Therefore, the input focal length of the optical lens system and the reading position of the image sensor 6 do not always coincide with the focal length recorded in the table 300 and the coordinates P ′ (x ′, y ′).

In such a case, in the present embodiment, for example, the distortion correction coordinates are acquired as follows.
First, the distortion correction coordinate recording unit 11 reads, from the table 300, a focal length that is closest to the focal length of the input optical lens system among the focal lengths greater than the focal length of the input optical lens system.
Then, the distortion correction coordinate recording unit 11 has coordinates P ′ (x ′, y ′) that are registered in the table 300 in association with the read focal length and that are closest to the reading position of the image sensor 6. The distortion correction coordinates P (x, y) associated with '(x', y ') are read out.
Similarly, the distortion correction coordinate recording unit 11 reads, from the table 300, a focal distance that is closest to the focal length of the input optical lens system among the focal lengths smaller than the focal length of the input optical lens system.

Then, the distortion correction coordinate recording unit 11 has coordinates P ′ (x ′, y ′) that are registered in the table 300 in association with the read focal length and that are closest to the reading position of the image sensor 6. The distortion correction coordinates P (x, y) associated with '(x', y ') are read out.
The distortion correction coordinate recording unit 11 performs an interpolation process on the two distortion correction coordinates P (x, y) read out as described above, and the distortion correction coordinates P (x, y) at the reading position of the image sensor 6. Is output to the distortion correction unit 8.

In the table 300 shown in FIG. 3, the focal length measurement number n is a predetermined number n ₀ (n is a natural number), the focal length measurement interval is 5 mm, and the number of data at each focal length is three. Two cases are shown as examples. However, the setting method of the table 300, such as the number of data at each focal length and the setting of the focal length, is not limited to that shown in FIG.

The distortion correction unit 8 temporarily records the image signal output from the analog front end 7 in a memory such as a RAM (not shown). The image signal temporarily recorded in this manner is distorted due to the influence of distortion. Therefore, the distortion correction unit 8 temporarily recorded in the memory so that the coordinates of the image signal distorted due to the distortion aberration are replaced with the distortion correction coordinates output from the distortion correction coordinate recording unit 11. A read address for the image signal is generated. The distortion correction unit 8 of the present embodiment corrects the influence of distortion aberration generated in the image signal output from the analog front end 7 by changing the read address according to the corresponding distortion correction coordinates in this way. To do.
As described above, the distortion correction unit 8 outputs to the image recording unit 12 and the contour extraction unit 13 an image signal in which the influence of distortion is corrected by performing the two-dimensional coordinate conversion as described above.
The contour extraction unit 13 binarizes the image signal output from the distortion correction unit 8 to “0” or “1” according to a predetermined threshold, and extracts the contour information in the image signal from the binarized information. To the previous field memory 14 and the current field memory 15.

FIG. 5 is a diagram illustrating an example of an access area to the field memory of the motion vector calculation unit 16.
As shown in FIG. 5A, the previous field memory 14 records an image signal 501 of the entire screen delayed by one field. The current field memory 15 records an image signal 502 in a partial area of the screen as shown in FIG. Here, the image signal 502 shown in FIG. 5B is an image signal of the first block (the block shown by 1 in the figure) among the 16 blocks shown in FIG.
The motion vector calculation unit 16 cuts out the image signal 503 having the size of the area of the first block (image signal 502) from the image signal 501 recorded in the previous field memory 14 (see FIG. 5C). ). Thereafter, the motion vector calculation unit 16 sets a range 504 wider than the region of the first block (image signal 502) in order to calculate the motion vector of the region of the first block (image signal 502). Then, the correlation value of the image signals 502 and 503 is calculated within the set range 504 to obtain a motion vector in the first block. The motion vector calculation unit 16 performs the same process as the first block for the second to sixteenth blocks to obtain the motion vectors of the second to sixteenth blocks in order, and obtains the obtained first to sixteenth blocks. The motion vector of the entire screen is obtained using the motion vector of the block.

  FIG. 6 is a flowchart illustrating an example of motion vector detection operation. In this example, the period from the falling edge of the vertical synchronizing signal to the next falling edge is defined as one field period, and the operation starts from START in FIG. 6 at the beginning of one field period.

In step S1 of FIG. 6, the current field memory 15 records an image signal (for example, the image signal 502 of the first block) of a partial area of the screen (any of the blocks shown in FIG. 5F). .
In step S <b> 2, the motion vector calculation unit 16 determines the size of the area of the image signal (for example, the image signal 502 of the first block) recorded in the current field memory 15 from the image signal recorded in the previous field memory 14. Image signal (for example, image signal 503) is cut out. Then, the motion vector calculation unit 16 reads the clipped image signal.

  Next, in step S3, the motion vector calculation unit 16 reads the image signal recorded in the current field memory 15 in step S1, and reads the image signal read from the current field memory 15 and the image signal read from the previous field memory 14. And compare. Then, the motion vector calculation unit 16 calculates a correlation value of each image signal. When obtaining the motion vector of the first block, the motion vector calculation unit 16 compares the image signals 502 and 503 and calculates the correlation value of the image signals 502 and 503 with respect to the direction of the arrow shown in FIG.

  Next, in step S <b> 4, the motion vector calculation unit 16 sequentially changes the position where the image signal is cut out from the previous field memory 14. That is, the motion vector calculation unit 16 repeats the processes of steps S2 to S4 for a range that is wider than the image signal read from the current field memory 15 in step S1 and is set in advance. When obtaining the motion vector of the first block, as shown in FIG. 5E, the motion vector calculation unit 16 repeatedly performs the processes of steps S <b> 2 to S <b> 4 for a preset range 504.

And if the process of step S2-S4 is performed about the preset range, it will progress to step S5 and the motion vector calculating part 16 will have the highest correlation value among the several correlation values calculated by step S3. For example, the vector is recorded in a RAM (not shown).
Next, in step S <b> 6, the motion vector calculation unit 16 instructs the current field memory 15 to change the area to be recorded in the current field memory 15. Then, the motion vector calculation unit 16 repeatedly performs the processes of steps S1 to S6 for the first to sixteenth blocks set in advance as shown in FIG.

When the processes of steps S1 to S6 are completed for the first to sixteenth blocks set in advance as shown in FIG. 5F, the process proceeds to step S7. In step S7, the motion vector calculation unit 16 calculates the motion vector of the entire screen using the vector value in each block recorded in step S5.
In the present embodiment, motion vectors are obtained for 16 blocks, but it goes without saying that the number of blocks is not limited to 16.

  The address control unit 17 controls a position to be cut out from the image recording unit 12 according to the motion vector detected by the motion vector calculation unit 16. That is, the address control unit 17 corrects image output blur (so-called camera shake) by moving the cutout position of the screen in the direction opposite to the motion vector detected by the motion vector calculation unit 16.

Here, an example of the operation when creating the table 300 as shown in FIG. 3 will be described with reference to the flowchart of FIG. The operation illustrated in FIG. 7 is performed by a CPU (not illustrated) that is provided in an external information processing apparatus or imaging apparatus and that controls the entire apparatus while using a RAM (not illustrated) as a work memory. Can be realized.
First, in step S11, the number n of focal length measurements in the table 300 is set to one. Next, in step S12, the test image 201 at the nth focal length is acquired, and in step S13, the output image 202 of the imaging device at the nth focal length is acquired.

In step S14, the coordinates P (x, y) on the test image 201 and the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging apparatus corresponding to the coordinates P (x, y). And ask.
Next, in step S15, the obtained coordinates P (x, y) and P ′ (x ′, y ′) are associated with the nth focal length and registered in the table 300.
Next, in step S16, it is determined whether or not the number n of focal length measurements is a predetermined number n ₀ . That is, as shown in FIG. 3, it is determined whether or not all the coordinates P (x, y) and P ′ (x ′, y ′) to be registered have been registered for n ₀ focal lengths. As a result of the determination, if the focal length measurement number n is not the predetermined number n ₀ , the process proceeds to step S17. In step S17, 1 is added to the focal length measurement number n. Then, until the measurement number n of the focal length becomes a predetermined number n _0, it repeats the steps S12 to S17.

Next, an example of operations of the distortion correction coordinate recording unit 11 and the distortion correction unit 8 when correcting the influence of distortion occurring in the image signal will be described with reference to the flowchart of FIG.
First, in step S21, the distortion correction coordinate recording unit 11 stands by until the focal length of the optical lens system is input from the zoom encoder 9. When the focal length of the optical lens system is input from the zoom encoder 9, the process proceeds to step S <b> 22, and the distortion correction coordinate recording unit 11 waits until the readout position of the image sensor 6 is input from the timing generator 10. Note that the order of steps S21 and S22 may be reversed.

  When the focal length of the optical lens system and the reading position of the image sensor 6 are thus input, the process proceeds to step S23. In step S23, the distortion correction coordinate recording unit 11 inputs the input focal length of the optical lens system and the readout position of the image sensor 6 to the focal length and coordinates P ′ (x ′, y ′) registered in the table 300, respectively. ). As a result of this determination, if the input focal length of the optical lens system and the readout position of the image sensor 6 match the focal length and coordinates P ′ (x ′, y ′) registered in the table 300, respectively. The process proceeds to step S24, and if not, the process proceeds to step S25.

In step S24, the distortion correction coordinate recording unit 11 calculates the distortion correction coordinates P (x, y) associated with the focal length and the coordinates P ′ (x ′, y ′) registered in the table 300. Read from the table 300 and output to the distortion correction unit 8.
On the other hand, when the process proceeds to step S25, the distortion correction recording unit 11 performs the interpolation process as described above, and outputs the interpolation value obtained as a result to the distortion correction unit 8 as distortion correction coordinates.

  When the distortion correction coordinates are output to the distortion correction unit 8, the process proceeds to step S26. In step S26, the distortion correction unit 8 temporarily records the image signal output from the analog front end 7 in a memory such as a RAM (not shown). In addition, you may perform the process of this step S26 before step S24, S25.

Next, in step S27, the distortion correction unit 8 temporarily changes the coordinates of the image signal distorted due to the influence of the distortion aberration to the distortion correction coordinates output in step S24 or step S25. A read address of the recorded image signal is generated. Then, the distortion correction unit 8 reads the temporarily recorded image signal according to the generated read address, and outputs it to the image recording unit 12 and the contour extraction unit 13.
Next, in step S <b> 28, the distortion correction coordinate recording unit 11 determines whether all the readout positions of the image sensor 6 are input from the timing generator 10. If all the readout positions of the image sensor 6 are not input as a result of this determination, Steps S22 to S28 are repeated until all the readout positions of the image sensor 6 are input.

  As described above, in the present embodiment, the focal length of the optical lens system, the coordinates P (x, y) on the test image 201, and the coordinates P (x ′, y ′) on the output image 202 of the imaging device are calculated. They are recorded in advance in the table 300 in association with each other. When imaging starts, the distortion correction coordinate recording unit 11 inputs the focal length of the optical lens system from the zoom encoder 9 and inputs the readout position of the imaging element 6 from the timing generator 10.

Then, the distortion correction coordinate recording unit 11 acquires the distortion correction coordinates using the input focal length of the optical lens system, the reading position of the image sensor 6, and the table 300, and outputs it to the distortion correction unit 8. The distortion correction unit 8 corrects the influence of distortion generated in the image signal by changing the read address of the image signal output from the analog front end 7 in accordance with the distortion correction coordinates. The motion vector detection unit 16 detects a motion vector using the image signal in which the distortion correction unit 8 corrects the influence of distortion of the optical lens system.
As described above, in this embodiment, since the motion vector is detected using the image signal in which the influence of the distortion of the optical lens system is corrected, an error in detecting the motion vector due to the influence of the distortion of the optical lens system is detected. Judgment can be prevented as much as possible. Thereby, the detection accuracy of a motion vector can be improved.

  In the present embodiment, the motion vector is detected using the image signal in which the influence of the distortion of the optical lens system is corrected, and the camera shake is corrected by the address control unit 17 according to the detected motion vector. . The method of using the detected motion vector is not limited to this. That is, if the process uses a motion vector, it is not always necessary to correct camera shake. For example, a motion vector may be used to change a distance measurement area where AF (Auto Focus) is performed. By doing in this way, a ranging area can be changed appropriately. In addition, the motion vector of the subject may be detected.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, the case has been described in which the motion vector is accurately detected by the imaging apparatus in which the optical lens system is integrated. On the other hand, in this embodiment, a motion vector is accurately detected by an imaging device that can exchange the optical lens system. That is, in this embodiment, a distortion coefficient for correcting distortion is recorded in the optical lens system, and the distortion aberration generated in the image signal is calculated by using the distortion coefficient and the coordinate position of the image signal. This is to correct the influence.

  As described above, the present embodiment and the first embodiment described above include a method for generating data for correcting whether the optical lens system can be removed and the influence of distortion occurring in the image signal. Mainly different. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals as those in FIGS.

FIG. 9 is a diagram illustrating an example of the configuration of the imaging apparatus according to the present embodiment. As shown in FIG. 9, the image pickup apparatus 701 can be attached with an interchangeable lens 702.
In FIG. 9, reference numeral 18 denotes a distortion coefficient recording unit that records a distortion coefficient, and 19 denotes distortion correction coordinates using the distortion coefficient recorded in the distortion coefficient recording unit 18 and the readout position of the image signal. It is a distortion correction coordinate calculation unit. Reference numeral 20 denotes an imaging device side communication terminal, and reference numeral 21 denotes an interchangeable lens side communication terminal. The distortion coefficient recorded in the distortion coefficient recording unit 18 is input to the distortion correction coordinate calculation unit 19 via the imaging device side communication terminal 20 and the interchangeable lens side communication terminal 21.

  In the distortion coefficient recording unit 18, a distortion coefficient corresponding to the focal length is recorded in advance. The distortion coefficient recording unit 18 can be realized by using, for example, a ROM (not shown).

Here, an example of a method for calculating a distortion coefficient recorded in advance in the distortion coefficient recording unit 18 will be described.
First, when assembling a product in a factory, a test image 201 as shown in FIG. Then, a distortion coefficient corresponding to the focal length is calculated using the captured test image 201 and the output image 202 of the imaging apparatus as shown in FIG.

FIG. 10 illustrates the distance from the center of the test image 201 corresponding to the optical center to the coordinates of an arbitrary point on the test image 201 and the arbitrary image on the output image 202 of the imaging device from the test image 201 corresponding to the optical center. It is a figure which shows an example with the distance to the coordinate of this point.
In FIG. 10, the coordinates of the test image 201 corresponding to the optical center are O (0, 0). In addition, the coordinates of an arbitrary point on the test image 201 are P (x, y) as ideal coordinates without distortion. Furthermore, the coordinates of an arbitrary point on the output image 202 of the imaging apparatus are P ′ (x ′, y ′) as coordinates distorted by distortion. In addition, the distance from the center of the test image 201 corresponding to the optical center to the coordinates of an arbitrary point on the test image 201 is r, and from the center of the test image 201 corresponding to the optical center to the output image 202 of the imaging device. The distance to the coordinates of an arbitrary point is set as r ′.

Here, the distortion rate D (r) [%] at the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging apparatus is expressed by Expression (1).
D (r) = (r′−r) / r × 100 (1)
The distortion is affected as the distance from the optical center O (0, 0) increases, and the characteristic changes depending on the focal length of the optical lens system.
In general, since the distortion rate D (r) is proportional to the square of the distance r from the center of the test image 201 corresponding to the optical center to the coordinates of an arbitrary point on the test image 201, α and β are determined by the focal length. Assuming that the constant is determined, the distortion rate D (r) can be approximated as shown in Equation (2).
D (r) = α × r ² (2)
Further, the relationship between the constants α and β depending on the focal length is represented by Expression (3).
β = 100α (3)

From Expressions (1), (2), and (3), the distance r ′ from the optical center to the coordinates on the output image 202 of the imaging apparatus is expressed by Expression (4).
r ′ = r (1 + β · r ² ) (4)
From the equation (4), the coordinate P ′ (x ′, y ′) on the output image 202 of the imaging device is (1 + β · r ² ) times optical as compared to the coordinate P (x, y) on the test image 201. It is considered that the test image 201 corresponding to the center is away from the coordinates O (0, 0). Accordingly, the horizontal and vertical distances are also separated by (1 + β · r ² ) times.

From Expression (4), the relationship between the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging apparatus and the coordinates P (x, y) on the test image 201 is expressed by Expressions (5) and (6). ).
x ′ = x (1 + β · r ² ) = x {1 + β · (x ² + y ² )} (5)
y ′ = y (1 + β · r ² ) = y {1 + β · (x ² + y ² )} (6)
The distortion coefficient β is calculated by using the test image 201, the output image 202 of the imaging device, and the equations (5) and (6). Such a distortion coefficient β is calculated for each focal length. A table in which the calculated focal length and the distortion coefficient β are associated with each other is recorded in advance in the distortion coefficient recording unit 18.
FIG. 11 is a diagram showing an example of recorded contents recorded in the table 1100 provided in the distortion coefficient recording unit 18. As shown in FIG. 11, in this embodiment, the focal length and the distortion coefficient β are recorded in the table 1100 at every predetermined focal length (for example, every 5 mm as shown in FIG. 11). This is because recording capacity increases if the distortion coefficient β is recorded for all focal lengths in the optical lens system. The table 1100 provided in the distortion coefficient recording unit 18 can be realized by using, for example, a flash memory (not shown).

The timing generator 10 outputs the read position information of the image sensor 6 to the distortion correction coordinate calculation unit 19.
The distortion correction coordinate calculation unit 19 transmits a distortion coefficient request signal to the distortion coefficient recording unit 18 via the imaging device side communication terminal 20 and the interchangeable lens side communication terminal 21 in order to acquire the distortion coefficient β. Then, the distortion coefficient recording unit 18 reads out the distortion coefficient β corresponding to the focal length of the optical lens system from the distortion coefficient recording unit 18, and performs distortion via the imaging device side communication terminal 20 and the interchangeable lens side communication terminal 21. It outputs to the correction coordinate calculation part 19.

  The distortion correction coordinate calculation unit 19 substitutes the distortion coefficient β output from the distortion coefficient recording unit 18 and the readout position information output from the timing generator 10 into Equations (5) and (6) to correct distortion. The coordinates P (x, y) are calculated and output to the distortion correction unit 8.

  As described above, in this embodiment, the distortion coefficient β is recorded in the table 1100 for each predetermined focal length. Therefore, the focal length of the input optical lens system does not always match the focal length recorded in the table 1100.

In such a case, in the present embodiment, the distortion coefficient β is obtained as follows.
First, when a distortion coefficient request signal is input from the distortion correction coordinate calculation unit 19, the distortion coefficient recording unit 18 has the largest focal length of the input optical lens system among the focal lengths greater than the input focal length of the optical lens system. Read the near focal length from the table 1100. Then, the distortion coefficient recording unit 18 reads the read focal distance and the distortion coefficient β registered in the table 1100 in association with the focal distance.

  Similarly, the distortion coefficient recording unit 18 reads, from the table 1100, a focal length that is closest to the focal length of the input optical lens system among the focal lengths smaller than the focal length of the input optical lens system. Then, the distortion coefficient recording unit 18 reads the read focal distance and the distortion coefficient β registered in the table 1100 in association with the focal distance.

The distortion coefficient recording unit 18 performs an interpolation process using the two distortion coefficients β read out as described above, and outputs the distortion coefficient β obtained as a result to the distortion correction coordinate calculation unit 19.
In the table 1100 shown in FIG. 11, the number n of focal lengths is a predetermined number n ₁ (n is a natural number), and the focal length measurement interval is 5 mm as an example. The setting of the focal length is not limited to that shown in FIG.

  The distortion correction unit 8 temporarily records the image signal output from the analog front end 7 in a memory such as a RAM (not shown). The image signal temporarily recorded in this manner is distorted due to the influence of distortion. Therefore, the distortion correction unit 8 is a memory such as a RAM (not shown) so that the coordinates of the image signal distorted due to the distortion aberration are replaced with the distortion correction coordinates output from the distortion correction coordinate calculation unit 19. The readout address of the image signal temporarily recorded in the is generated. The distortion correction unit 8 corrects the influence of distortion occurring in the image signal output from the analog front end 7 by changing the read address in accordance with the corresponding distortion correction coordinates in this way. Then, the distortion correction unit 8 outputs an image signal in which the influence of distortion is corrected to the image recording unit 12 and the contour extraction unit 13.

Here, an example of an operation when creating the table 1100 as shown in FIG. 11 will be described with reference to the flowchart of FIG. The operation illustrated in FIG. 12 is provided in, for example, an external information processing apparatus or imaging apparatus, and a CPU (not illustrated) that controls the entire apparatus executes a program while using a RAM (not illustrated) as a work memory. Can be realized.
First, in step S31, the number n of focal length measurements in the table 1100 is set to one. Next, in step S32, the test image 201 at the nth focal length is acquired, and in step S33, the output image 202 of the imaging device at the nth focal length is acquired.

In step S34, the coordinates P (x, y) on the test image 201 and the coordinates P ′ (x ′, y ′) on the output image 202 of the imaging apparatus corresponding to the coordinates P (x, y). And ask.
Next, in step S35, the obtained coordinates P (x, y) and P ′ (x ′, y ′) are substituted into equations (5) and (6) to calculate the distortion coefficient β.
Next, in step S36, the distortion coefficient β and the nth focal length are associated with each other and registered in the table 1100.
Next, in step S37, it is determined whether or not the number n of focal length measurements is a predetermined number n ₁ . That is, as shown in FIG. 11, it is determined whether or not the distortion coefficient β has been registered for all focal lengths to be registered in the table 1100. As a result of the determination, if the focal length measurement number n is not the predetermined number n ₁ , the process proceeds to step S38. In step S38, 1 is added to the focal length measurement number n. Then, steps S32 to S38 are repeated until the focal length measurement number n reaches the predetermined number n ₁ .

Next, referring to the flowchart of FIG. 13, the distortion coefficient recording unit 18, the distortion correction coordinate calculating unit 19, and the distortion correction when calculating the distortion correction coordinates and correcting the influence of the distortion occurring in the image signal. An example of the operation of the unit 8 will be described.
First, in step S <b> 41, the distortion correction coordinate calculation unit 19 waits until the readout position of the image sensor 6 is input from the timing generator 10. Next, in step S <b> 42, the distortion correction coordinate calculation unit 19 transmits a distortion coefficient request signal to the distortion coefficient recording unit 18.
Next, in step S <b> 43, the distortion coefficient recording unit 18 determines whether or not the input focal length of the optical lens system matches the focal length registered in the table 1100. As a result of this determination, if the focal length of the input optical lens system matches the focal length registered in the table 1100, the process proceeds to step S44, and if not, the process proceeds to step S45.

In step S 44, the distortion coefficient recording unit 18 reads the distortion coefficient β associated with the focal length registered in the table 1100 from the table 1100 and outputs it to the distortion correction coordinate calculation unit 19.
On the other hand, when the process proceeds to step S45, the distortion coefficient recording unit 18 performs the interpolation processing as described above, and outputs the interpolation value obtained as a result to the distortion correction coordinate calculation unit 19 as the distortion coefficient β.

When the distortion coefficient β is output to the distortion correction coordinate calculation unit 19, the process proceeds to step S46. In step S46, the distortion correction coordinate calculation unit 19 substitutes the input readout position of the image sensor 6 and the distortion coefficient β output from the distortion coefficient recording unit 18 into the equations (5) and (6), and performs distortion. Correction coordinates P (x, y) are calculated and output to the distortion correction unit 8.
Next, in step S47, the distortion correction unit 8 temporarily records the image signal output from the analog front end 7 in a memory such as a RAM (not shown). Note that the processing in step S47 may be performed before step S46.

Next, in step S48, the distortion correction unit 8 replaces the coordinates of the image signal distorted under the influence of the distortion aberration with the distortion correction coordinates P (x, y) calculated in step S46. A read address for the temporarily recorded image signal is generated. Then, the distortion correction unit 8 reads the temporarily recorded image signal according to the generated read address, and outputs it to the image recording unit 12 and the contour extraction unit 13.
Next, in step S <b> 49, the distortion correction coordinate calculation unit 19 determines whether or not all the readout positions of the image sensor 6 are input from the timing generator 10. If all the readout positions of the image sensor 6 are not input as a result of this determination, Steps S41 to S49 are repeated until all the readout positions of the image sensor 6 are input.

As described above, in the present embodiment, the distortion correction coordinate P (x, y) is calculated using the distortion coefficient β of the optical lens system, the reading position of the image sensor 6 and the equations (5) and (6). I tried to do it. Therefore, as shown in FIG. 3, it is possible to acquire the distortion correction coordinates with a recording medium having a small capacity compared to the case of recording the distortion correction coordinates P (x, y) corresponding to all the reading positions. (See FIG. 11).
In addition, since distortion information is provided in the optical lens system itself, distortion correction corresponding to a lens used in an interchangeable lens type digital video camera can be performed.

In this embodiment, the distortion coefficient recording unit 18 is provided in the interchangeable lens 702. However, the distortion coefficient recording unit 18 may be provided in the imaging device 701.
Further, in the present embodiment, the configuration in which the interchangeable lens 702 is attached to the imaging device 701 has been described as an example. However, as in the first embodiment, the imaging device is an integrated optical lens system. In addition, it is possible to correct distortion using the distortion coefficient β described in the present embodiment. In this case, for example, in the configuration shown in FIG. 9, the optical lens system and the distortion coefficient recording unit 18 may be provided in the imaging device 701.

Further, in the present embodiment, the interpolation processing is performed by the distortion coefficient recording unit 18, but the interpolation processing may be performed by the distortion correction coordinate calculation unit 19.
Furthermore, even the imaging apparatus capable of exchanging the optical lens system as in this embodiment can realize the first embodiment described above. In this case, for example, in the configuration shown in FIG. 9, the table 300 shown in FIG. 3 is provided instead of the distortion coefficient recording unit 18, and the distortion correction coordinate recording unit is used instead of the distortion correction coordinate calculation unit 19. 11 may be provided (configuration other than the table 300).

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the first and second embodiments described above, the distortion correction unit 8 corrects the influence of the distortion occurring on the entire screen when obtaining the motion vector. On the other hand, in the present embodiment, the influence of the distortion aberration is corrected only for the area necessary for the current field memory for detecting the motion vector. That is, in the present embodiment, by using the second distortion correction unit 22 that corrects the influence of the distortion only for the area required in the current field memory, the value of the motion vector after the image is formed on the image sensor 6. This shortens the time until detection. In the present embodiment, the motion vector detection result is fed back and the shake is optically corrected.

  As described above, the present embodiment and the first and second embodiments described above mainly include a method for correcting the influence of distortion, a method for using a motion vector detection result, and a method for correcting a shake of an imaging apparatus. Different. Therefore, in the description of the present embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals as those in FIGS.

Before describing the present embodiment, as a comparison with the imaging apparatus of the present embodiment, first, a motion vector detection operation in the imaging apparatus of the first embodiment will be described.
FIG. 14 is a diagram illustrating an example of operation timing of the imaging apparatus according to the first embodiment.
In FIG. 14, the period from the fall of the vertical synchronizing signal to the next fall is defined as one field period, the current field is F0, the field before F is F-1, and the field after is F1.

  FIG. 14A shows the output of the image sensor 6. An image signal for one screen is read from the image sensor 6 in one field period, digitized by the analog front end 7 and output to the distortion correction unit 8. Here, the image signal read in the current field is 0, the image signal read before 1 field is -1, and the image signal read after 1 field is 1.

FIG. 14B shows the output of the distortion correction unit 8.
The distortion correction unit 8 requires distortion correction processing time for one field in order to correct distortion of the entire screen area of the image sensor 6.
Therefore, as shown in FIG. 14B, the distortion correction unit 8 outputs an image signal delayed by one field from the output of the image sensor 6.
FIG. 14C shows the output of the previous field memory 14.
Since the front field memory 14 has a delay of one field with respect to the input signal, the front field memory 18 outputs an image signal delayed by two fields from the image sensor 6 as shown in FIG. To do.

FIG. 14D shows the motion vector detection timing.
Here, the field (field F-1) indicated by the number shown on the left side of the arrow ("-1" in FIG. 14) and the field (field F0) indicated by the number shown on the right side of the arrow ("0" in FIG. 14). It shows that the motion vector between is detected.

The motion vector calculation unit 16 operates using the output signal of the distortion correction unit 8 shown in FIG. 14B as the current field and the output signal of the previous field memory 18 shown in FIG. Vector detection is performed.
For this reason, the motion vector “−1 → 0” generated in the field F 0 that is the current field period is detected as the next field F 1. As a result, in the imaging apparatus according to the first embodiment, when processing is performed by feeding back the motion vector detection result, there is a disadvantage that the response speed decreases. Therefore, in this embodiment, such an inconvenience is solved. Hereinafter, the imaging apparatus of the present embodiment will be described.

FIG. 15 is a diagram illustrating an example of the configuration of the imaging apparatus according to the present embodiment.
In FIG. 15, 22 is a second distortion correction unit, and 23 is a second contour extraction unit. Reference numeral 24 is a variable apex angle prism (hereinafter referred to as VAP), and 25 is a VAP controller. Note that the first distortion correction unit 8 and the first contour extraction unit 13 are the same as the distortion correction unit 8 and the contour extraction unit 13 shown in FIG. 1, respectively.

Similar to the first distortion correction unit 8, the second distortion correction unit 22 includes an image signal digitized by the analog front end 7 and distortion correction coordinates P (x read out from the distortion correction recording unit 11. , Y).
Thereafter, like the first distortion correction unit 8, the second distortion correction unit 22 temporarily records the image signal input from the analog front end 7 in a memory such as a RAM (not shown). Then, the second distortion correction unit 20 includes a RAM so that the coordinates of the image signal temporarily recorded in the memory are replaced with the distortion correction coordinates P (x, y) input from the distortion correction coordinate recording unit 11. A read address of the image signal temporarily recorded in the memory is generated.

  In this way, by changing the read address in accordance with the corresponding distortion correction coordinates P (x, y), the second distortion correction unit 22 causes the distortion generated in the image signal output from the analog front end 7. Correct the effects of aberrations. Then, the second distortion correction unit 22 outputs an image signal in which the influence of distortion is corrected by performing the two-dimensional coordinate conversion as described above to the second contour extraction unit 23.

The second contour extraction unit 23 binarizes the image signal output from the second distortion correction unit 22 into “0” or “1” according to a predetermined threshold value, and generates an image from the binarized information. The contour information in the signal is extracted and output to the current field memory 15.
Here, the output of the first distortion correction unit 8 is an image signal used as an image output. Therefore, the first distortion correction unit 8 needs to perform distortion correction on the entire area of the image.

  On the other hand, the second distortion correction unit 22 uses an image signal in an area necessary for the current field memory 15 used for motion vector detection (for example, in a partial area of the screen shown in FIG. 5B). The image signal 502) is corrected in order. That is, the second distortion correction unit 22 corrects the image signal in order for each block shown in FIG. Then, the motion vector calculation unit 16 calculates a motion vector in order for each block whose image signal is corrected by the second distortion correction unit 22.

The distortion correction processing time by two-dimensional coordinate transformation as described above is proportional to the number of correction pixels. For this reason, the second distortion correction unit 22 can perform distortion correction in a shorter time than the first distortion correction unit 8.
As described above, in this embodiment, the image signal from the first distortion correction unit 8 is the image signal of the previous field, and the image signal from the second distortion correction unit 22 is the image signal of the current field. Detection is performed.

  The VAP control unit 25 generates a VAP control signal based on the motion vector of the image signal detected by the motion vector calculation unit 16. The VAP 22 optically corrects the shake by changing the apex angle according to the VAP control signal.

A motion vector detection operation in the imaging apparatus of the present embodiment will be described with reference to FIGS. 15 and 16. FIG. 16 is a diagram illustrating an example of operation timing of the imaging apparatus according to the present embodiment.
Similarly to FIG. 14, in FIG. 16, the period from the falling edge of the vertical synchronizing signal to the next falling edge is defined as one field period, the current field is F0, the field before F-1 is, and the field after F1 is F1. Will be described.

FIG. 16A shows the output of the image sensor 6. An image signal for one screen is read from the image sensor 6 in one field period, digitized by the analog front end 7 and output to the first and second distortion correction units 8 and 22. Here, the image signal read in the current field is “0”, the image signal read before one field is “−1”, and the image signal read after one field is “1”.
FIG. 16B shows the output of the second distortion correction unit 20.
The second distortion correction unit 22 performs distortion correction on only the pixel area necessary for the current field memory 15 for the image signal input from the analog front end 7.
Here, the second distortion correction unit 22 can output an image signal to the second contour extraction unit 23 in the same field as the output of the image sensor 6 by reducing the pixel region to be subjected to distortion correction.

FIG. 16C shows the output of the first distortion correction unit 8.
The first distortion correction unit 8 corrects the distortion of the entire area of the image. For this reason, one field of distortion correction processing time is required.
FIG. 16D shows the motion vector detection timing.
Here, the motion vector between the field indicated by the number shown on the left side of the arrow and the field indicated by the number shown on the right side of the arrow is detected.

The motion vector calculation unit 16 uses the output signal from the second distortion correction unit 22 shown in FIG. 16B as the current field signal, and outputs the signal from the first correction unit 8 shown in FIG. The motion vector is detected using the output signal as a signal one field before.
Therefore, the motion vector “0 → 1” generated in the field F0 which is the current field period is detected in the current field F0. As a result, even when a motion vector detection result is fed back, control with a good response speed can be performed.

In the present embodiment, not only when the distortion correction coordinates are recorded in the distortion correction coordinate recording unit 11 as in the first embodiment, but also in the distortion correction coordinates using the distortion coefficient β as in the second embodiment. This can also be applied to the case where the distortion correction coordinate calculation unit 19 calculates.
In the present embodiment, as in the first embodiment and the second embodiment, the shake may be corrected by changing the cutout position of the screen. Furthermore, there is a possibility that the response speed may be slow, but in the first embodiment and the second embodiment, it may be configured to optically correct shake as in the present embodiment.

(Other embodiments of the present invention)
In order to operate various devices to realize the functions of the above-described embodiments, program codes of software for realizing the above-described functions are supplied to an apparatus or a computer in the system connected to the various devices. Are also included in the scope of the present invention. And what was implemented by operating various devices according to the program stored in the computer (CPU or MPU) of the system or apparatus is also included in the category of the present invention.

  In this case, the software program code itself realizes the functions of the above-described embodiments. The program code itself and means for supplying the program code to the computer, for example, a recording medium storing the program code constitute the present invention. As a recording medium for storing such a program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, a DVD, or the like can be used.

  The program code is also included in the embodiment of the present invention when the function of the above-described embodiment is realized in cooperation with an OS (operating system) or other application software running on the computer. It is.

  Further, the supplied program code is stored in a memory provided in a function expansion board of the computer or a function expansion unit connected to the computer. Thereafter, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. It goes without saying that it is included in the present invention.

  It should be noted that the above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

1 is a diagram illustrating an example of a configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the 1st Embodiment of this invention and shows an example of a test image and the output image of an imaging device. It is a figure which shows the 1st Embodiment of this invention and shows an example of the recording content of the table provided in the distortion correction coordinate recording part. It is a figure which shows the 1st Embodiment of this invention and shows an example of the ideal coordinate and distortion coordinate without a distortion aberration. It is a figure which shows the 1st Embodiment of this invention and shows an example of the access area to a memory. It is a flowchart which shows the 1st Embodiment of this invention and shows an example of the operation | movement of motion vector detection. 5 is a flowchart illustrating an example of an operation when creating a table in the distortion correction coordinate recording unit according to the first embodiment of this invention. 12 is a flowchart illustrating an example of an operation when correcting the influence of distortion occurring in an image signal according to the second embodiment of this invention. It is a figure which shows the 2nd Embodiment of this invention and shows an example of a structure of an imaging device. The second embodiment of the present invention, the distance from the center of the test image corresponding to the optical center to the coordinates of an arbitrary point on the test image, and the output image of the imaging device from the test image corresponding to the optical center It is a figure which shows an example with the distance to the coordinate of arbitrary upper points. It is a figure which shows the 2nd Embodiment of this invention and shows an example of the recording content recorded on the table provided in the distortion coefficient recording part. 10 is a flowchart illustrating an example of an operation when creating a table in the distortion correction recording unit according to the second embodiment of this invention. FIG. 9 is a flowchart illustrating an example of an operation when correcting the influence of distortion occurring in an image signal by calculating distortion correction coordinates according to the second embodiment of the present invention. FIG. 3 is a diagram illustrating an example of operation timing of the imaging apparatus according to the first embodiment of this invention. It is a figure which shows the 3rd Embodiment of this invention and shows an example of a structure of an imaging device. FIG. 10 is a diagram illustrating an example of operation timing of the imaging apparatus according to the third embodiment of this invention. It is a figure which shows notionally the mode that the image shift produced by the distortion aberration. It is a figure which shows notionally a mode that a to-be-photographed image deform | transforms with an imaging position.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 4 Fixed lens 2 Zoom lens 3 Aperture 5 Focus lens 6 Image sensor 7 Analog front end 8 Distortion correction part 9 Zoom encoder 10 Timing generator 11 Distortion correction coordinate recording part 12 Image recording part 13, 21 Contour extraction part 14 Previous field memory 15 Current field memory 16 Motion vector calculation unit 17 Address control unit 18 Distortion coefficient recording unit 19 Distortion correction coordinate calculation unit 20 Imaging device side communication terminal 21 Interchangeable lens side communication terminal 22 Second distortion correction unit 23 Second contour extraction Part 24 Variable vertical angle prism (VAP)
25 VAP controller

Claims (12)

An image sensor that converts an object image formed by an optical lens that forms an image on an imaging surface into an electrical signal; and
Calculation means for calculating distortion correction data for correcting distortion of the optical lens;
Distortion correction means for correcting distortion aberration of the optical lens with respect to an image based on a subject image converted into an electrical signal by the imaging device based on distortion aberration correction data calculated by the calculation means;
A motion vector detection means for detecting a motion vector using an image in which the distortion is corrected by the distortion correction means;
Motion correction means for correcting motion based on the detection of the motion vector ,
The image pickup apparatus , wherein the distortion correction unit corrects distortion aberration of the optical lens only in a region necessary for detecting a motion vector .   The calculation means, from the table for correcting the influence of distortion of the optical lens, the distortion correction coordinates corresponding to the focal length of the optical lens and the coordinates of the image affected by the distortion of the optical lens The image pickup apparatus according to claim 1, wherein the distortion aberration correction data is calculated using the acquired distortion aberration correction coordinates.   The calculation means acquires a distortion coefficient corresponding to the focal length of the optical lens from a table for correcting the influence of the distortion aberration of the optical lens, and calculates distortion aberration correction data using the acquired distortion coefficient. The imaging apparatus according to claim 1, wherein:   The imaging apparatus according to claim 3, wherein the distortion coefficient is a coefficient depending on a distortion rate of the optical lens and a distance from a center of the image.   The imaging apparatus according to claim 2, further comprising the table. Further comprising communication means for communicating with the interchangeable lens device having the table;
The calculating means, the pre Kiibitsu songs aberration correction data, to any one of claims 2 to 4, characterized in that calculated using the information of the table in which the interchangeable lens unit has The imaging device described. An image sensor that converts an object image formed by an optical lens that forms an image on an imaging surface into an electrical signal; and
Calculation means for calculating distortion correction data for correcting distortion of the optical lens;
Distortion correction means for correcting distortion aberration of the optical lens with respect to an image based on a subject image converted into an electrical signal by the imaging device based on distortion aberration correction data calculated by the calculation means;
A motion vector detection means for detecting a motion vector using an image in which the distortion is corrected by the distortion correction means;
Motion correction means for correcting motion based on detection of the motion vector;
Communication means for communicating with an interchangeable lens apparatus having a table for correcting the influence of distortion of the optical lens,
The calculation means corresponds to the focal length of the optical lens and the coordinates of an image affected by distortion of the optical lens from the table of the interchangeable lens device via the communication means. An image pickup apparatus, comprising: acquiring distortion aberration correction coordinates to be calculated, and calculating the distortion aberration correction data using the acquired distortion aberration correction coordinates. An image sensor that converts an object image formed by an optical lens that forms an image on an imaging surface into an electrical signal; and
Calculation means for calculating distortion correction data for correcting distortion of the optical lens;
Distortion correction means for correcting distortion aberration of the optical lens with respect to an image based on a subject image converted into an electrical signal by the imaging device based on distortion aberration correction data calculated by the calculation means;
A motion vector detection means for detecting a motion vector using an image in which the distortion is corrected by the distortion correction means;
Motion correction means for correcting motion based on detection of the motion vector;
Communication means for communicating with an interchangeable lens apparatus having a table for correcting the influence of distortion of the optical lens,
The calculation means obtains a distortion coefficient corresponding to the focal length of the optical lens from the table of the interchangeable lens apparatus via the communication means, and uses the obtained distortion coefficient to obtain distortion aberration. An image pickup apparatus that calculates correction data. The image pickup apparatus according to claim 8, wherein the distortion coefficient is a coefficient depending on a distortion rate of the optical lens and a distance from a center of an image. An imaging method using an imaging device including an imaging element that converts an object image formed by an optical lens that forms an image on an imaging surface into an electrical signal,
A calculation step of calculating distortion correction data for correcting distortion of the optical lens;
A distortion correction step of correcting distortion aberration of the optical lens with respect to an image based on a subject image converted into an electric signal by the imaging element based on the distortion aberration correction data calculated by the calculation step;
A motion vector detection step of detecting a motion vector using the image in which the distortion is corrected by the distortion correction step;
A motion correction step of correcting motion based on the detection of the motion vector ,
The imaging method, wherein the distortion correction step corrects distortion aberration of the optical lens only in a region necessary for detecting a motion vector . A computer program for causing a computer to execute each step of the imaging method according to claim 10 . A computer-readable storage medium storing the computer program according to claim 11 .

Images