JPS59183588A - Method for correcting distortion of lens - Google Patents

Abstract

PURPOSE:To correct distortion of a lens without causing color shear by performing the correction of distortion of the lens not singly but together with color separation in an electronic still camera. CONSTITUTION:When an optical image is formed on a photodetecting face of an image pickup element, a grating of straight line is formed as a grating of pincushion form due to distortion of lens. For example, optical information to be made cucident substantially to a picture element C3 is made incident to upper left near a picture element D4. In this case, since the color separation is performed at the same time with the correction of distortion and the light to have been made incident to the C3 is made incident to the upper left of the D4 in calculating R,G,B of the picture element of the C3,C4,D3,D4 are selected as approximate picture elements and the calculation is performed by using M, G, Cy and Y. The color separation is performed by 4 picture elements closest to the positions taking the offset due to distortion into account in this way, allowing to correct distortion without causing color shear.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a lens distortion correction method that prevents color shift that occurs when lens distortion is corrected by image signal processing.

In a patent application titled "Electronic Still Camera" filed on the same date as the present application, the present inventors converted two-dimensional image information from a subject into an image signal using an image sensor, and converted this image signal into an
An electronic still camera has been proposed which performs -D conversion and stores the converted digital image information in a removable storage means. Since this electronic still camera digitally processes image signals, it can obtain high-quality still images without signal deterioration during signal processing.Also, since the digital image data is memorized for a month, a rotating mechanism is not required and power consumption is reduced. It is superior to conventional electronic still cameras in many ways, including the ability to reduce noise, make the camera body more compact, and improve reliability. Figure 1 shows the basic circuit configuration of this electronic still camera, in which 1 is an optical lens that forms an optical image of the subject;
is an image sensor that photoelectrically converts an optical image (two-dimensional image information) formed on a light-receiving surface by an optical lens l (two-dimensional image information) and outputs it as an analog image signal. , Flambicon, Cosbicon, New Cosbicon, and other image pickup tubes and CCDs.

Examples include solid-state imaging devices such as MOS and CPD.

The light-receiving surface of the image sensor 2 (this is where the color filter 2a is pasted so that a color image can be obtained.The image sensor 2 is
An analog image signal is output according to a drive signal from the driver 3, and is converted into a digital signal at high speed by the image signal converter 4. In this case, the digitization level is determined by the required image quality, but about 8 bits are required for general television images.The sampling rate is generally color subcarrier (358 MF).
z), but this is not necessarily the case when the imaging section of an electronic still camera has a temporary storage function.

The image signal digitally converted by the A/D converter 4 is stored in the memory 5. The memory 5 can be anything as long as it can store digital signals stably.
N-MOS RAM.

In addition to semiconductor memories such as C-MOS RAM and CCD memory, magnetic bubble elements, magnetic disks, magnetic tapes, etc. are used. Writing and reading of image signals into and from the memory 5 are controlled by a driver 6. The memory 5 is, for example, in the form of a cassette and is replaceable and can be attached to and removed from the camera body at a predetermined position from the outside. Of course, it is necessary to back up the memory (or batteries, etc.) by putting it inside the cassette.

On the other hand, the digital image signal output from the A/D converter 4 is directly or once stored in the memory 5 and then converted into an analog image signal by the D/A converter 7. The image is then sent to a monitor CRT (displayed as two images).

In this case, the selector switch 8 is set by the driver 6 by external operation.
Contacts a and b are switched by a switching signal from. In addition, the output of the image sensor 2 is directly connected to the changeover switch 8 (contact a).
It is also possible to visualize the image using the electronic finder 9 via the electronic viewfinder 9. In this case, since the output of the image sensor 2 is modulated by the color filter 2a, when the image is displayed directly on the electronic viewfinder 9, it is different from a normal black and white image, but it is difficult to know what is being displayed. can.

Note that 10 is a battery that serves as a power source for each of the circuit elements described above.

FIG. 2 is a block diagram of one embodiment of the player, in which the memory 5 is removed from the camera and set at a predetermined position in the player (indicated by a broken line). In the reproducing machine, the microcomputer 12 reads the video data recorded in the memory 5 according to the software in the ROM 131 in the storage means 13, and processes the data using the RAM 132 as a work area. Processed data is written to the field memory 14 according to instructions from the microcomputer 12;
2, it is always driven by the TV rate drive circuit 15, and its output is connected to the D/
The 1'N images are displayed on the monitor via the A converter 16. As a result, the recorded video gradually appears on the monitor 17.

In the above-mentioned electronic still camera, an optical image of the subject is formed on the light-receiving surface of an image sensor by an optical lens, and is photoelectrically converted and output as an analog image signal. When using a solid-state image sensor, there is a phenomenon in which the image formation point at the periphery of the lens is distorted due to lens distortion. Figure 3 shows this situation. The ideal (
Due to lens distortion, the straight line that should be imaged (as shown by the thick line) is distorted (as shown by the broken line). Therefore, for example, the image should normally be formed at the pixel position C5. image is formed at the pixel position B5, and if the image signal obtained in this way is reproduced as is, the original linear grid pattern will be displayed as a distorted grid pattern shown by the broken line.The degree of this distortion is varies depending on the lens.

Therefore, in order to eliminate such lens distortion, conventional efforts have been made at the lens manufacturing stage, such as adjusting the diameter and thickness of the lens or the curvature of the lens surface, or increasing the number of combined lenses. Processing is time-consuming, resulting in higher costs, larger lens diameters, and heavier weights.

For this reason, the reality is that the lenses of recent video cameras and electronic still cameras are extremely large compared to the camera body.

Therefore, the present inventor proposed a distortion correction method in which the distortion of the lens in an electronic still camera is corrected by electrical processing of the image signal, regardless of the manufacturing techniques of the lens. For example, the distortion information of the optical lens of the electronic still camera used during the shooting process is stored together with the image signals for one screen, and the image signals are rearranged in the storage area of the playback device based on the distortion information. Here, the lens distortion information is stored in the same way as the method of storing color separation information disclosed in "Electronic still camera with color separation information stored" filed on the same day (this method is used to store information in empty areas of the memory and in scans that do not appear in the image). Lens distortion information can be written to lines, horizontal or vertical retrace sections, and can be retrieved by the playback device.For example, as shown in Figure 4, the linear grid (solid line) that is the subject Considering the case where the image is formed as a pincushion-like grid (dashed line) due to lens distortion, the image signal memo IJ & is stored in the arrangement shown in Table 1 corresponding to each pixel. However, in order to correct lens distortion, each pixel is rubbed and arranged as shown in Table 2. The pattern appears on playback.
2. However, in the case of color photography, the image signal is color-separated by a color filter provided on the front of the image sensor and stored in memory as color separation information for each pixel, so the method described above is used during playback. When lens distortion is corrected, the color information of the image signal for the read pixel becomes different from the original color information. For example, if a color filter with a color configuration as shown in FIG. 5 is used in an image sensor having an image arrangement as shown in FIG. 3 or 4, pixel AI is M (magenta) and pixel A2 is G ( Green), A3 is M% A4 is G.

...B1 is Cy (cyan), B2 is Y (yellow), B3 is q, ..., CI is M, C2 is G,
C3 is color-modulated with M, C4 with G, and so on. Therefore, in the example shown in Fig. 4, if the image signal of pixel B4 is read out as the image information of pixel A4 in order to correct lens distortion, the image signal of pixel A4 is originally color-modulated by G of the color filter. The color information that should be the correct color information is read out as color information modulated in Y, and the color information is confused due to distortion correction, and the colors of the reproduced image are no longer faithful to the colors of the subject. There is a problem with this.

The present invention solves the problem of color misalignment when correcting lens distortion. By correcting lens distortion at the same time as color separation, instead of correcting lens distortion independently, lens distortion can be corrected without color misalignment. The intention is to make corrections.

The present invention will be explained below based on the drawings.

FIG. 6 shows an example in which a color filter having the color configuration shown in FIG. 5 is used in an image sensor having a pixel arrangement as shown in FIG. In this example, the linear grating (solid line) is imaged as a pincushion-shaped grating (dashed line) due to lens distortion, so for example, the light information color-modulated by the color filter that should originally be incident on pixel C3 is transmitted to pixel D4. The beam is incident diagonally upward and to the left near . If color separation is performed using an analog signal processing circuit in this state, for example, R, G, B of pixel C3
is B2. B3. C2゜C3, Y, cy, G
, M is calculated using M.

With this method, no color shift occurs in areas other than the edges, but distortion is not corrected. In the present invention, color separation and distortion correction are performed simultaneously, so when calculating R,..., G, and B of pixels of C3, the light that should be incident on C3 is incident diagonally above and to the left of C4. Taking this into consideration, we selected C3, C4, C3, C4 as the closest four pixels, and set M, G here.
.

Calculate using Cy and Y. If the incident position is C
If it is directly above 4 instead of diagonally to the left, you can use the average value of C3, C5, or the average value of C3, C5 instead of C3° C3, or if it is directly above, use the left (right) ,
C3, C3 (C5°C5) may also be used. Eventually (as shown above, if you choose to perform color separation using the four pixels closest to the position that takes into account deviations due to distortion, you can perform distortion correction without causing color deviations. Also, the range of distortion correction The memory area required to perform this process can be reduced by limiting the amount of time required for this process.For example, the deviation of the incident position due to lens distortion is within 0 (maximum scanning lines in the sub-scanning direction) (this is quite a requirement for the lens). If this is the case, then the RAM area should be enough for original data and 13 scanning lines for R, G, and B.
, G, and B, all that is left to do is to encode them and write them into the playback memory, so no other QRAM is required. For this purpose, the camera's storage means (indicated by 5 in Figure 1)
Even if it is a magnetic disk or magnetic tape, it can be easily handled by a playback machine.

If such distortion correction is performed at the same time as color separation, it is possible to perform distortion correction without causing color shift, and furthermore, with a small memory work area, it can be compatible with the camera-side storage means of all types of electronic still cameras. .

[Brief explanation of the drawing]

FIG. 1 is a schematic diagram of the configuration of an electronic still camera to which the present invention is applied, FIG. FIG. 5 is a diagram showing an example of the color configuration of a color filter used in an electronic still camera; FIG. 6 is an explanatory diagram for explaining the color shift prevention method according to the present invention. l...Optical lens 2...Image sensor 3.6...
Driver 4-A/D converter 5...Memory
7, 6...D/A converter 8...Switch 9...Electronic finder 10...Battery
12...Microcomputer 13...Storage means-131...-ROM. 132...RAM 15...Drive circuit 17
... Monitor patent applicant Konishiroku Photo Industry Co., Ltd. Attorney Hiroo Suzuki No. 2 Figure 3 Figure 4 @ Figure 5 Figure 6

Claims (1)

[Claims] In a lens distortion correction method in which light from a subject is incident on an image sensor through an optical lens and a color filter, and an image signal from the image sensor is corrected in accordance with the distortion of the optical lens, when color separation is performed, A method for correcting lens distortion, characterized in that image signals referenced by color pixels are selected in consideration of deviations due to lens distortion, and the image is corrected.