JP3523660B2 - Imaging device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and more particularly to an image pickup apparatus for masking an unnecessary image portion displayed by distortion correction. 2. Description of the Related Art Since an optical lens has aberrations, an optical distortion of a subject optical image formed on an image pickup device through the optical lens occurs, and as a result, a video signal also has a distortion. Image. As the optical distortion, FIG.
There is a "threading type distortion" as shown in FIG. 11A and a "barrel type distortion" as shown in FIG. These distortions are shown in FIG.
In No. 1, the distortion is such that image information that should originally be at the position indicated by the dotted line forms an image at the solid line position. [0003] As a correction process for correcting such a distortion of a video signal accompanied by optical distortion, a video signal is converted into a digital signal, written into an image memory, and read addresses are shifted in accordance with the distortion to read out the image memory. There is a process for correcting optical distortion above. For example, in FIG. 12, if there is no distortion due to the optical lens, it is assumed that a grid-like image to be stored as a dotted line in the image memory is stored as a solid line due to optical distortion. When this image data is read from the image memory, the image data stored at point a at the time when point A is to be read, the image data stored at point b at the time when point B is to be read, and The image data stored at the point c is read at the timing when the data is to be read. By doing so, the distorted image shown by the solid line is read out as the original lattice-like image without the distortion shown by the dotted line, and the optical distortion is corrected. FIG. 13 is a block diagram showing a configuration example of a conventional image pickup apparatus having this kind of correction function. A subject image is formed on an image sensor 2 such as a CCD via an optical system 1 such as an optical lens. The image formed on the image sensor 2 includes the optical distortion, and is converted into an electric signal by the image sensor 2. The signal from the image sensor 2 is subjected to a predetermined process in the image sensing process circuit 3 and converted into an A /
It is supplied to the D converter 4. The video signal converted into a digital signal by the A / D converter 4 is stored in the image memory 5. The timing of writing and reading signals to and from the image memory 5 is controlled by a write control circuit 10 and a read control circuit 12. A synchronization signal generation (SSG) circuit 9 generates a reference timing signal for the operation of the device, and generates a timing generator (TG) circuit 8, an imaging process circuit 3, and a light control circuit 1.
Supply 0. The TG circuit 8 sends readout timing signals in the horizontal (H) direction and the vertical (V) direction from the SSG circuit 9 to the image sensor 2. Light control circuit 10
Is an image memory 5 for a video signal from the A / D converter 4
Control the write timing to the The microcomputer 11 receives signals such as zoom information (focal length information of the zoom lens) from the optical system 1 and corrects the optical distortion based on the correction amount data stored in the correction amount ROM 17. The read control circuit 12 is controlled. The correction amount ROM 17 stores, for each lens use condition, a correction amount determined in advance for each part of the screen, for example, a correction amount determined based on a relationship between a solid line position and a dotted line position in FIG. In this way, the signal read from the image memory 5 to correct the optical distortion based on the read signal output from the read control circuit 12 is subjected to the interpolation processing by the interpolation circuit 6, and then converted into an analog signal by the D / A converter 7. It is converted and output. An imaging device having such an optical distortion correction function is disclosed in Japanese Patent Application Laid-Open No. 4-61570. [0006] When a video signal obtained by the processing of the conventional image pickup apparatus is displayed on a monitor, when a "barrel distortion" as shown in FIG. As shown in FIG. 14B, an image without distortion is displayed.
However, when a part of the image of the “pincushion distortion” as shown in FIG. 14C is exposed outside the effective imaging area of the imaging element, image information outside this effective area cannot be obtained. The hatched portion of the image portion (D) outside the effective imaging area is not displayed on the monitor screen. As described above, when a part of the image including the distortion is formed outside the effective imaging area, the correction is performed without obtaining the image information outside the effective imaging area, so that the part of the image is displayed on the monitor screen. There is a problem that the image cannot be projected and the display screen becomes unsightly. SUMMARY OF THE INVENTION It is an object of the present invention to provide an image pickup apparatus in which a portion other than an original image projected by distortion correction is masked to remove the unsightly display screen. [0008] To solve the above problems BRIEF SUMMARY OF THE INVENTION The imaging apparatus according to the present invention, an image an image signal obtained by imaging the imaging device via an optical system to be Utsushitai image A correction means for storing the image information in the storage means, and controlling the writing or reading of image information to or from the image storage means to correct the distortion caused by the optical system; and an image signal projected by the image signal created by the correction means. Masking means for performing a masking process on at least an area where image information is not obtained outside the effective imaging area of the image sensor on the screen, wherein the masking means is configured to perform the distortion regardless of a change in the focal length of a zoom lens. Is configured to perform a masking process in the corresponding mask area when the distortion rate of the mask is maximum. According to the present invention, an image signal is stored in a memory, and when the distortion caused by the optical system is corrected by controlling the writing and reading of the image signal to and from the memory, the image is projected by the corrected image signal. Masked areas on the screen where no image information can be obtained , and the masking processing
Does not depend on the change in the focal length of the zoom lens.
This is performed in the corresponding mask area when the read ratio is maximum. Next, an embodiment of the present invention will be described with reference to the drawings. In FIG. 1, components denoted by the same reference numerals as those in FIG. 13 indicate components having the same functions. FIG. 2A shows an example of an optical distortion characteristic with respect to a relationship between a relative distance (%) from an optical axis and a distortion rate D (%) in a zoom lens. Here, the horizontal axis represents the length of a half of the diagonal line of the effective surface of the imaging device as 10
The relative distance from the optical axis position at 0% is shown, and the vertical axis shows the distortion rate D. Here, the definition of the distortion rate D is defined as follows: when an image to be formed at a point at a relative distance r is formed at r ′ due to optical distortion, as shown in FIG. 2B, D = (r′− r) / r × 100% (1) Although the characteristics fluctuate depending on the focal length f, the distortion ratio D increases as r increases, and this characteristic can be approximated by, for example, D = s ″ · r ² (2). From the equations (1) and (2), r ′ = r (1 + s ′ · r ² ) (3) Here, s ″ and s ′ are coefficients determined by the focal length, and s ′ = s ″ / 100. That is, according to equation (3), an image to be formed at a point at a relative distance r from the optical axis on the image sensor is formed at a point at (1 + s' · r ² ) times r due to optical distortion. Can lead. Considering a point P in the memory which is at a relative distance r from the optical axis on the image sensor, as shown in FIGS. 3A and 3B, for example, in the case of an NTSC signal, the aspect ratio of the image sensor is obtained. Is about 3: 4, and the case where the video signal is stored in a 240 × 768 field memory is considered. If the relative distance r on the image sensor is, for example, R pixels in the horizontal direction on the memory, the relative distance r is R / 2.4 pixels in the vertical direction, and is represented by different numbers of pixels in the horizontal direction and the vertical direction. Therefore, in this example, the number of pixels in the vertical direction is set to 2.
By multiplying by a conversion coefficient k such as 4, the relative distance r on the image sensor in the horizontal and vertical directions can be converted into the number of pixels in the memory, such as R pixels in the memory. Here, as shown in FIG. 4, a point to be imaged at a point P at a relative distance r from the optical axis on the image sensor is formed at a point P 'at a relative distance r' due to optical distortion. Think about it. For point P, the position of x pixels in the horizontal direction and y pixels in the vertical direction from the center coordinates of the memory corresponding to the optical axis on the image sensor. For point P ', x' pixels in the horizontal direction and y 'pixels in the vertical direction. P, P 'on the memory at the position
Find the positional relationship of From equation (3), point P ′ is (1+
It is considered that the distance from the center coordinate is s ′ · r ² ) times. Here, when r is represented by a size on a memory, it can be represented by r = c × {x ² + (ky) ² } (4). c is a constant determined by the size of the image sensor and the number of pixels in the memory. Further, the fact that the point P ′ is (1 + s ′ · r ² ) times away from the center coordinate of the point P means that the distance in the horizontal and vertical directions is also (1 + s ′ · r ² ) times. x ′ = x (1 + s ′ · r ² ) (5) y ′ = y (1 + s ′ · r ² ) (6) From equations (4), (5) and (6), x ′ = x [1 + s ′ · c ² ｛x ² + (ky) ² ｝] (7 ′) y ′ = y [1 + s ′ · c ² ｛x ² + (Ky) ² }] (8 ′) Here, if s′C ² is put together with a constant s, x ′ = x [1 + s {x ² + (ky) ² }] (7) y ′ = y [1 + s} x ² + (ky) ² }] (8) S is a coefficient determined by the focal length. As is apparent from the above equation, the pixel P (x, x, y, x, y in the horizontal and vertical directions from the center coordinates of the memory
The image data to be stored in y) is a pixel P ′ (x ′) distant from the x ′, y ′ center coordinates of {1 + s (x ² + (ky) ² )) times x and y respectively due to optical distortion. , Y ')
Is stored. Therefore, as described above (in the conventional example), when the image data is read from the memory, the optical distortion is corrected by reading the image data stored at the point P ′ at the timing when the point P should be read. In the above-described embodiment, if the distance r from the optical axis on the image sensor is represented by pixels x and y of the memory, a square root operation such as r = √ (x ² + y ² ) is entered by the square theorem. In this embodiment, the optical distortion characteristic is expressed by D as shown in the equation (2).
= S ″ · r ² , so that the square root operation and the square operation cancel each other, and the scale of the arithmetic circuit is reduced. In particular, the calculation of the square root requires a large circuit scale, so that the effect is large. FIG. 5 shows a read control circuit 12 for generating a read address for correcting the optical distortion.
Is shown. The horizontal read timing address from the H counter 121 is
From 2, a read timing address in the vertical direction is generated. The read timing of the image memory 5 is the same as that of the television scanning, and is read from the upper left to the lower right. When the coordinate system is set as shown in FIG.
0), the lower right is (2x ₀ , 2y ₀ ), and the center is (x ₀ , y ₀ ). Equations (7) and (8) consider the center coordinate of the image memory 5 as the origin, whereas the H counter 121 and V
The read timing address supplied from the counter 122 has the origin at the upper left as shown in FIG.
Therefore, it is necessary to move the origin so that the supplied address is distance information from the center coordinate address (x ₀ , y ₀ ). The origin moving block circuit 123
A subtraction circuit 123 for executing such origin movement, subtracting the center coordinate address values x ₀ and y ₀ from the address values from the H counter 121 and the V counter 122, respectively.
1 and 1232. As a result of the origin moving process, the coordinate system becomes a coordinate system as shown in FIG. Next, equation (7) is used in the distance calculation block 124.
And the distance calculation in (8): x ² + (ky) ² . After the input x is squared by the multiplier 1241, the adder 1
242, the input y is multiplied by a transform coefficient k by a multiplier 1243, then squared by a multiplier 1244, and the outputs of the multipliers 1241 and 1244 are added by an adder 1242. The distortion magnification calculation block 125 is given by the following equation (7).
And [8] are used to calculate [1 + s {x ² + (ky) ² }].
Multiplied by the focal length coefficient s determined by the focal length of the zoom lens is supplied from ^{{x 2 + (ky) 2} } to the microcomputer 11 supplied from the adder 1251 adds "1" to the multiplication calculated force Output. X ′ and y ′ in equations (7) and (8) are x ′,
It is obtained by the multipliers 1261 and 1262 of the y 'operation block 126. Multipliers 1261 and 1262 multiply the output of adder 1252 by x and y output from subtractors 1231 and 1232, respectively. The thus obtained x 'and y' are the addresses of the xy coordinates when the center is the origin as shown in FIG. 2B, and the actual origin of the image memory 5 is the upper left as described above. Therefore, the adders 1271 and 1272 of the origin moving block 127 add the x 'and y' center coordinate address values, respectively, to return the coordinates as shown in FIG. Through the above processing, the image memory 5
A read address corresponding to the image distortion is generated, and by reading the image memory 5 with this read address, an image with optical distortion corrected can be obtained. The optical distortion characteristics in the description of the above embodiment can be approximated by various equations, and the higher the order, the higher the accuracy. In the above embodiment, the optical distortion is corrected by the read control of the memory, but it is needless to say that this may be performed by the write control of the memory. Referring now to FIG. 1, the microcomputer 11
Calculates the distortion correction coefficient from the focal length information of the zoom lens of the optical system 1 such as an optical lens and calculates the distortion correction coefficient.
Feed to 2. The read control circuit 12 controls the read address of the image memory 5 based on the distortion correction coefficient supplied from the microcomputer 11 and the horizontal / vertical synchronization signal supplied from the SSG circuit 9, and as the interpolation data of the interpolation circuit 6. Is calculated. The mask address generator 15 outputs a range to be masked as mask address data in a memory space as described later. The mask control circuit 14
The read address from the read control circuit 12 is compared with the mask address data from the mask address generator 15, and a control signal for controlling the switch 13 is output as described later. Data generator 16 for mask
Outputs image data of a mask unit as described later.
The switch 13 outputs video data corrected for distortion based on a control signal from the mask control circuit 14 or mask video data which is an output from the mask data generator 16. For example, if the masking data is pedestal level data, a part of the video data is black, and if it is an arbitrary color level data, a part of the video data is masked so as to have an arbitrary color. . FIG. 6 shows a circuit configuration example of the mask control circuit 14. As shown in FIG. 7, when the row address of the image memory 5 is x and the column address is y, the video data is (x ₀ , y ₀ ), (x ₁ , y ₀ ), (x ₀ , y ₁ ), (X
₁ , y ₁ ) is controlled and stored by the light control circuit 10 in a rectangular area having vertices. The switch 13 outputs the signal High from the OR gate 145 of the mask control circuit 14 to High.
If h, mask image data is selected, and if Low, distortion-corrected video is selected and output. As shown in FIG. 6, the output x ′ of the read control circuit 11 is input to the − terminal of the comparator 141 and the + terminal of the comparator 142, and the output y ′ is input to the − terminal of the comparator 143 and the + terminal of the comparator 144.
Input to terminal. On the other hand, the output x ₀ ′ of the mask address generator 15 is connected to the other terminal of the comparator.
x ₁ ′, y ₀ ′, and y ₁ ′ are input. The comparator outputs High when the input of the + terminal is larger than the input of the − terminal, and outputs Low otherwise. FIG. 8 shows a truth table of the input / output relationship of the mask control circuit 14. Here, the output of the mask address generator 15 is represented by x ₀ ′ = x ₀ , x
Assuming that ₁ ′ = x ₁ , y ₀ ′ = y ₀ , y ₁ ′ = y ₁ , a mask is applied when accessing an address where no image is stored according to the truth table of FIG. As shown on the screen. In the case of masking as shown in FIG. 10, when the distortion rate of the pincushion distortion is the maximum, the position of the object light imaged in the square of the image sensor when there is no distortion is to be formed. Is converted to the memory space by (x ₂ ,
y _2), and _{(x 3, y 2),} (x 2, y 3), (x 3, y 3). The output x ₀ ′, x ₁ ′,
Let y ₀ ′, y ₁ ′ be x ₀ ′ = x ₂ , x ₁ ′ = x ₃ , y ₀ ′ = y ₂ ,
y ₁ ′ = y ₃ and the read control circuit 11
Is output not from the read address (x ′, y ′) after the distortion correction operation processing but from the H.0 in the circuit. V. By replacing the address (x, y) from the counter, the mask can be performed as shown in FIG. Since the distortion rate is the maximum, even if the focal length of the zoom lens changes, there is no problem since the portion without image information falls within the mask area. As described above, according to the present invention,
When a distortion-corrected image is projected on the screen, the area where unsightly parts occur on the screen is masked by the usual distortion correction means, so that discomfort during shooting or monitoring can be reduced. Have. Also the focal length
Mask area during the masking process
No additional processing is required, and processing can be simplified.
You.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration block diagram illustrating an embodiment of an imaging apparatus according to the present invention. FIG. 2 is a diagram illustrating an example of optical distortion characteristics of a zoom lens for describing an operation of the embodiment of the present invention. FIG. 3 is a diagram for explaining the operation of the embodiment of the present invention. FIG. 4 is a diagram for explaining the operation of the embodiment of the present invention. FIG. 5 is a detailed configuration block diagram of a read control circuit 12 in the embodiment shown in FIG. FIG. 6 is a detailed configuration block diagram of a mask control circuit 14 in the embodiment shown in FIG. FIG. 7 is a diagram for explaining the operation of the mask control circuit 14 in the embodiment shown in FIG. 8 is a diagram showing, in a truth table, an input / output relationship of the mask control circuit 14 in the embodiment shown in FIG. 1; FIG. 9 is a diagram showing a display example after performing a mask process according to the embodiment of the present invention. FIG. 10 is a diagram showing another display example subjected to mask processing according to the embodiment of the present invention. FIG. 11 is a diagram illustrating an example of optical system distortion. FIG. 12 is a diagram for explaining correction of optical system distortion. FIG. 13 is a configuration block diagram of a conventional imaging device. FIG. 14 is a diagram for explaining a problem of a conventional imaging device. [Description of Signs] 1 Optical system 2 Imaging device 3 Imaging process circuit 4 A / D converter 5 Image memory 6 Interpolation circuit 7 D / A converter 8 TG circuit 9 SSG circuit 10 Write control circuit 11 Microcomputer 12 Read control circuit 13 Switch 14 Mask control circuit 15 Mask address generator 16 Mask data generator 17 Correction amount ROM

Continuation of front page    (72) Inventor Hiroyuki Watanabe               2-43-2 Hatagaya, Shibuya-ku, Tokyo Oh               Linpass Optical Co., Ltd. (72) Inventor Dai Kawase               2-43-2 Hatagaya, Shibuya-ku, Tokyo Oh               Linpass Optical Co., Ltd. (72) Inventor Masaomi Tomizawa               2-43-2 Hatagaya, Shibuya-ku, Tokyo Oh               Linpass Optical Co., Ltd.                (56) References JP-A-4-23575 (JP, A)                 JP-A-59-66271 (JP, A)                 JP-A-59-196665 (JP, A)                 JP-A-4-61570 (JP, A)

Claims (1)

(57) [Claims] 1. An image of a subject is formed on an image sensor via an optical system.
The image signal obtained by this is stored in the image storage means,
Writing image information into this image storage means, or
Due to the optical system by controlling the reading
Correcting means for correcting distortion, and an image projected by an image signal created by the correcting means.
Effective imaging of at least the image sensor on a projected screen
Mask the area outside the area where image information cannot be obtained.
Masking means , wherein the masking means changes the focal length of the zoom lens.
Irrespective of the
That the mask area is configured to be masked
An imaging device characterized by the following.