JPH06237406A - Video processing unit - Google Patents

Abstract

PURPOSE:To simplify the control operation without causing a visual sense of incongruity by using the same correction coefficient when a change in distortion if an image corresponding to a change in a focal distance of an optical system is within a prescribed range. CONSTITUTION:A voltage detection section 111 in a microcomputer 11 detects a voltage corresponding to a zoom ratio of a zoom lens 1, a voltage/focal distance conversion section 112 converts the detected voltage into a focal distance, and a focal distance zone is discriminated. The zone indicates a division range being a range of a focal distance change corresponding to a range of a + or -1% distortion change. Then the presence of a zone revision is discriminated and when the zone is revised, a correction coefficient S is operated by a parameter setting section 113, the parameter is sent to a read address generating block, and when no zone revision is in existence, the coefficient S is used as it is. When the optical distortion is within a prescribed range and the image has no visual sense of incongruity, the same correction coefficient is used to save the memory capacity and the quantity of arithmetic operation required for correcting optical distortion thereby simplifying the control operation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device, and more particularly to an image processing device having an electronic zoom function, a camera shake correction function, and a function for correcting optical distortion such as an optical lens.

[0002]

2. Description of the Related Art Since an optical lens has an aberration, an optical distortion occurs in an optical image of a subject formed on an image pickup element through the optical lens, and as a result, a video signal is also distorted. turn into. The optical distortion is shown in FIG.
There is a "thread winding type distortion" as shown in (A) and a "barrel type distortion" as shown in FIG. 20 (B). These distortions are
At 0, the distortion is such that the image information originally supposed to be at the position indicated by the dotted line is imaged at the solid line position.

As a correction process for correcting the distortion of a video signal accompanied by such optical distortion, the video signal is converted into a digital signal and written in an image memory, and a read address is shifted according to the distortion to read out the image memory. There is a process for correcting the optical distortion above.

For example, in FIG. 21, if there is no distortion due to the optical lens, it is assumed that the lattice-shaped image to be stored in the image memory as shown by the dotted line is stored as shown by the solid line due to the optical distortion. When this image data is read from the image memory, the image data stored at the point a at the timing of reading the point A, the image data stored at the point b at the timing of reading the point B, the same as the point C. The image data stored at the point c is read at the timing at which is to be read. By doing so, the distorted image shown by the solid line is read as the original image without distortion shown by the dotted line, and the optical distortion is corrected.

FIG. 22 is a block diagram showing an example of the configuration of a conventional video processing device having this type of correction function.
A subject image is formed on an image sensor 2 such as a CCD via an optical system 1 such as an optical lens. The image formed on the image sensor 2 contains the optical distortion and is converted into an electric signal by the image sensor 2. The signal from the image pickup device 2 is subjected to a predetermined process in the image pickup process circuit 3 and supplied to the A / D converter 4 as a video signal. A / D converter 4
The video signal converted into a digital signal by the image memory 5
Memorized in. The timing of writing and reading signals to and from the image memory 5 is controlled by the write control circuit 10 and the read control circuit 12A. SS
The G circuit 9 generates a reference timing signal for the operation of the device and supplies it to the TG circuit 8, the imaging process circuit 3 and the write control circuit 10. The TG circuit 8 sends the read timing signals from the SSG circuit 9 in the horizontal (H) direction and the vertical (V) direction to the image sensor 2. The write control circuit 10 controls the timing of writing the video signal from the A / D converter 4 into the image memory 5.

The microcomputer 11 receives signals such as zoom information from the optical system 1 and controls the read control circuit 12A to correct the above optical distortion based on the correction amount data stored in the correction amount ROM 19. Correction amount ROM1
9 stores a correction amount that is predetermined for each part of the screen for each lens usage condition, for example, a correction amount that is determined by the relationship between the solid line position and the dotted line position in FIG. Thus, the signal read from the image memory 5 to correct the optical distortion by the read signal output from the read control circuit 12A is interpolated by the interpolating circuit 6 and then D / A.
It is converted into an analog signal by the converter 7 and output. An image processing device having such an optical distortion correction function is disclosed in Japanese Patent Application Laid-Open No. 4-61570.

[0007]

As described above, the conventional image processing apparatus stores the image signal having optical distortion in the image memory, and sets the correction amount for each pixel according to the optical distortion in advance.
It is stored in M and the optical distortion is corrected by the read address based on the correction amount read from the ROM according to the optical distortion.

In the image processing apparatus as described above, the optical distortion is defined by the focal length of the optical system and the distance from the optical axis of the image forming position. Therefore, if the focal length changes, the degree of correction also changes. There is a need. However, it is generally said that human beings can detect optical distortion when the distortion ratio is 1% or more. Therefore, it cannot be said that changing the correction state one by one according to the change of the focal length is suitable for the human visual characteristics. On the other hand, the control operation is very complicated, and the correction by the focal length detection error is rather made. There was a risk of wobbling.

In view of the above problems, the present invention is to provide an image processing apparatus of this type having an optical distortion correction function which has less visual discomfort and simplifies the control operation. .

[0010]

In order to solve the above-mentioned problems, a video processing apparatus according to the present invention converts a video signal obtained by photoelectrically converting an image by an optical system into video data corresponding to this video signal. An image processing apparatus comprising: a conversion unit; and a distortion correction unit that, when writing or reading the image data by the conversion unit to or from the storage unit, gives a correction corresponding to the image distortion of the optical system to the image data, The distortion correction means corrects the video signal with correction data as data to be applied for each focal length section in which a change in the degree of image distortion corresponding to a change in the focal length of the optical system is within a predetermined range. It is configured to include the data holding means as data that is the basis of processing.

[0011]

According to the present invention, the distortion correcting means for correcting the image data by the optical system to correct the distortion of the image by writing or reading the image data into the storage means, and the distortion correcting means of the image corresponding to the change of the focal length of the optical system. Correction data, which is data to be applied to a focal length section in which a change in distortion is within a predetermined range, is given as basic data for correction processing.

[0012]

Embodiments of the present invention will now be described with reference to the drawings. FIG. 1 shows a configuration block diagram of an example in which a processing circuit that performs an electronic zoom function and an optical distortion correction function is also used. In FIG. 1, components designated by the same reference numerals as those in FIG. 22 are components having the same function. In FIG.
The microcomputer 11 receives the designated electronic zoom magnification and sends the zoom information to the optical system 1 to perform optical zoom control, and at the same time, a zoom coefficient S indicating the zoom information and an H address pitch corresponding to the zoom magnification. , V address pitch is sent to the read control circuit 12. The read control circuit 12 sends signals to the image memory 5 and the interpolation circuit 6 for performing electronic zoom processing and optical distortion correction processing based on these pieces of information.

FIG. 2A shows an example of the optical distortion characteristics of the relationship between the relative distance (%) from the optical axis and the distortion rate D (%) in the zoom lens. Here, the abscissa represents the half length of the diagonal line of the effective surface of the image sensor to 10
The relative distance from the optical axis position when 0% is shown, and the vertical axis shows the strain rate D. Here, the definition of the distortion rate D is as follows. When an image to be formed at a point distant from the relative distance r is formed at r ′ due to optical distortion, as shown in FIG. 2B, D = (r′− r) / r × 100% (1) Although the characteristic varies depending on the focal length f, the distortion rate D increases as r increases, and this characteristic can be approximated by, for example, D = s ″ · r ² (2), and from Equations (1) and (2) r ′ = r (1 + s ′ · r ² ) (3). Here, s ″ and s ′ are coefficients determined by the focal length, and s ′ = s ″ / 100. That is, according to the equation (3), an image to be formed at a point distant from the optical axis on the image sensor by a relative distance r is imaged at a point distant by (1 + s ′ · r ² ) times r due to optical distortion. Can lead.

Considering the point P on the memory, which is a relative distance r from the optical axis on the image pickup device, as shown in FIGS. 3A and 3B, for example, in the case of an NTSC signal, the aspect ratio of the image pickup device is increased. Is about 3: 4, and the case where a video signal is stored in a 240 × 768 field memory will be considered. The relative distance r on the image pickup device is R / 2.4 pixels in the vertical direction, assuming that there are R pixels in the horizontal direction in the memory, and is represented by different numbers of pixels in the horizontal and vertical directions. Therefore, the number of pixels in the vertical direction is 2.
By multiplying by a conversion coefficient k such as 4, the relative distance r on the image sensor in both the horizontal and vertical directions can be converted to the number of pixels in the memory, such as R pixels on the memory.

Here, as shown in FIG. 4, a point to be imaged at a point P distant from the optical axis on the image pickup element by a relative distance r is imaged at a point P ′ at a relative distance r ′ due to optical distortion. Consider the case. For point P, the position of x pixels in the horizontal direction and y pixels in the vertical direction from the center coordinates of the memory corresponding to the optical axis on the image sensor, and for the point P ', x'pixels in the horizontal direction and y'pixels in the vertical direction. P, P'on the memory at the position of
Find the positional relationship of.

From the equation (3), the point P ′ is (1+
It is considered to be s ′ · r ² ) times away from the center coordinate. Here, r can be expressed as r = c × {x ² + (ky) ² } (4) when expressed by the size on the memory. c is a constant determined by the size of the image sensor and the number of pixels in the memory. Also, the fact that the point P ′ is (1 + s ′ · r ² ) times away from the center coordinates from the point P means that the distances in the horizontal and vertical directions are also (1 + s ′ · r ² ) times away. x ′ = x (1 + s ′ · r ² ) (5) y ′ = y (1 + s ′ · r ² ) (6) Equation (4), (5), from (6) x '= x [ 1 + s' · c 2 {x 2 + (ky) 2}] (7') y '= y [1 + s' · c 2 {x 2 + (Ky) ² }] (8 ′) Here, if s′c ² is put together with a constant S, x ′ = x [1 + S {x ² + (ky) ² }] (7) y ′ = y [1 + S { x ² + (ky) ² }] (8). Further, S is a coefficient determined by the focal length.

As is clear from the above equation, the pixel P (x, y, which is separated horizontally and vertically from the center coordinates of the memory by x, y.
The image data to be stored in y) is pixel P ′ (x ′) distant from the center coordinates of x ′ and y ′ times {1 + s (x ² + (ky) ² )} times x and y due to optical distortion. , Y ')
Is stored in. Therefore, as described above (in the conventional example), when the image data is read from the memory, the optical distortion is corrected by reading the image data stored at the point P ′ at the timing when the point P should be read.

In the above description, when the distance r from the optical axis on the image pickup device is represented by the pixels x and y of the memory, the square root calculation such as r = (x ² + y ² ) is entered by the theorem of 3 squares. Then, the optical distortion characteristic is given by D =
Since it is approximated by s ″ · r ² , the square root calculation and the square calculation cancel each other, and the scale of the calculation circuit is reduced. In particular, the calculation of the square root requires a large circuit scale, so that it is very effective.

FIG. 5 shows a circuit example of the read control circuit 12 in this example. In FIG.
First, in order to perform electronic zoom processing, H address pitch data from the microcomputer 11 (0.5 in the case of 2 × zoom)
And V address pitch data (0.
5) is supplied to the H address pitch generating circuit 1202 and the V address pitch generating circuit 1204, respectively. The H start address generation circuit 1201 and the V start address generation circuit 1203 are circuits for generating a processing start address. In this example, “0” is generated as a start address, and these addresses are latch circuits 1205.
And 1207 are latched. The modes of the H start address generation circuit 1201, the H address pitch generation circuit 1202, the V start address generation circuit 1203, and the V address pitch generation circuit 1204 are switched by the mode selection signal. The address pitch output from the H address pitch generation circuit 1202 and the V address pitch generation circuit 1204 is latched by the latch circuits 1206 and 1208. Latch circuits 1205, 1206, 1207 and 12
08 operates in response to a vertical (V) synchronizing pulse, and the data read from each latch circuit is input to one input terminal of each of the selection circuit 1209, the adder 1211, the selection circuit 1212, and the adder 1214. Supplied. The outputs of the adders 1211 and 1214 are supplied to the other input terminals of the selection circuits 1209 and 1212 which switch and output the inputs in response to the horizontal (H) sync pulse and the V sync pulse, respectively. That is, the selection circuits 1209 and 1212 output the data from the latch circuits 1205 and 1207 at the start, and switch and output the data from the adders 1211 and 1214 from the next address. Selection circuits 1209 and 12
12 outputs latch circuits 1210 and 1213, respectively.
Are latched by and the latch outputs are clocked by CLOC respectively.
It is supplied as the other input of the adder in response to the K and H sync pulses, and is output as a zoom output signal. Such a structure is disclosed in JP-A-2-250469.

With such a configuration, the H start address generation circuit 1201 and the V start address generation circuit 12
The start address (0,
From 0), the address in pitch unit determined according to the zoom magnification is output from the latch circuits 1210 and 1213. These latch circuits generate the read timing address in the horizontal direction and the read timing address in the vertical direction. The reading timing of the image memory 5 is the same as that of the scanning of the television, and the reading is performed from the upper left to the lower right. When the coordinate system is set as shown in FIG. 9A, the origin is (0, 0) in the upper left, (2x ₀ , 2y ₀ ) in the lower right, and the center is (x
₀ , y ₀ ).

In the equations (7) and (8), the center coordinates of the image memory 5 are considered as the origin, whereas the read timing addresses supplied from the latch circuits 1210 and 1213 are as shown in FIG. The upper left is the origin. Therefore, it is necessary to move the origin so that the supplied address becomes the distance information from the central coordinate address (x ₀ , y ₀ ). The origin movement block circuit A executes such origin movement, and includes the latch circuits 1210 and 12
Subtraction circuits 1215 and 121 for subtracting the center coordinate address values x ₀ and y ₀ from the address value from 13 respectively.
It consists of six. As a result of this origin movement processing, the coordinate system becomes a coordinate system as shown in FIG.

Next, in the distance calculation block B, the distance calculation in equations (7) and (8): x ² + (ky) ² is executed. The input x is squared by the multiplier 1217 and then added by the adder 1
The input y is input to the multiplier 218, the input y is multiplied by the conversion coefficient k in the multiplier 1219, and then squared in the multiplier 1220. The outputs of the multipliers 1217 and 1220 are added in the adder 1218.

The distortion magnification calculation block C has the following equation (7):
[1 + s {x in (8)²+ (Ky) ²}] Is calculated
In the circuit, from the distance calculation block B by the multiplier 1221
Supplied {x²+ (Ky)²} From the microcomputer 11
Focal length determined by the focal length of the zoom lens supplied
The separation coefficient S is multiplied, and the multiplication output is added in the adder 1222.
"1" is added to and output.

X'and y'in the equations (7) and (8) are x ',
It is obtained by the multipliers 1223 and 1224 of the y ′ operation block D. The multipliers 1223 and 1224 are the adders 12
The output of 22 is multiplied by x and y output from the subtracters 1215 and 1216, respectively.

The x'and y'obtained in this way are the addresses of the xy coordinates when the center is the origin, as shown in FIG. 7B, and as described above, the actual origin of the image memory 5 is the upper left. Therefore, the adder 1 of the origin movement block E is
At 225 and 1226, the coordinates are returned to their original positions by adding the x ', y'center coordinate address values, respectively, as shown in FIG.

Through the above processing, the image memory 5
A read address corresponding to the distortion of the image is generated, and if the image memory 5 is read at this read address, an image with the optical distortion corrected can be obtained.

The optical distortion characteristics in the above description can be approximated by various formulas, and can be approximated with higher accuracy by approximating higher order terms. In the above-described example, the optical distortion is corrected by the memory read control, but it goes without saying that the memory distortion may be corrected by the memory write control. In the above example, the example in which the distortion correction is performed after the electronic zoom process is performed has been described, but it goes without saying that the electronic zoom may be performed after the distortion correction is performed.

Next, an example in which the camera shake correction processing section and the optical distortion correction processing section are used in common will be described with reference to FIGS. 6 and 7. 6 and 7, the circuit components designated by the same reference numerals as those in FIG. 1 are similar circuit components. In this example, as shown in FIG. 6, for example, a motion detector 13 such as an angular velocity sensor for detecting the amount of camera shake correction is provided, and the obtained motion information is sent to the microcomputer 11 to read the read control circuit 12.
Then, camera shake correction and optical distortion correction processing as described below are performed.

In FIG. 7, horizontal motion correction data (H) corresponding to the amount of camera shake from the motion detector 13
And vertical motion correction data (V) are H start address generation circuit 1201 and V start address generation circuit 12
03, the start addresses HS and VS are latched by the latch circuits 1205 and 1207. H address pitch generation circuit 1202 and V address pitch generation circuit 12
Reference numeral 04 is for slightly zooming the image so as not to cause a displacement (angle of view cut) of the image from the prescribed region when a camera shake occurs. For example, a value of 0.9 is set as the pitch information HP and VP. Is output. As a result, the latch circuit 12
The data output from 10 and 1213 are H
S, HS + HP, HS + 2HP, ... and VS, VS + V
P, VS + 2VP, ... The subsequent processing is the same as in FIG. 5, and the camera shake correction and the optical distortion correction processing are performed at the same time.

As described above, in the above-described example, all circuits are common except for the read control circuit 12, but in the read control circuit, the address generation circuit section for camera shake correction or electronic zoom processing and optical distortion correction processing is independent. Therefore, when one of the functions is unnecessary, only one of them can be operated. Therefore, power saving can be achieved. According to this example, an image processing apparatus that is inexpensive and easy to miniaturize can be obtained.

By the way, as described above, the optical distortion is corrected by detecting the focal length information of the optical system and the distance information from the optical axis of the image forming position, and based on this detection information.
Generally, when an image signal is taken out by using a storage type photoelectric conversion element, the storage time of the subject light and the output time are different, and the distortion correction is performed by detecting the focal length information of the optical system during the output period of the video signal. If you do, the focal length information of the optical system during the accumulation period of the output video signal does not necessarily match,
There is an excess or deficiency in the correction. In addition, the focal length of the optical system changes every second, so if the focal length information of the optical system is detected in real time and distortion correction is performed, the amount of distortion correction will differ even within the same screen, resulting in unnatural images. turn into.

An example in which the above problem is solved by delaying the information of the optical system during the accumulation period of the output video signal until the reading timing of the corresponding image will be described. In this example, information such as the focal length of the optical system is fetched at a predetermined timing in the charge accumulation period of the image sensor, and the timing for giving the zoom coefficient S in the read address generator is set at the start of reading the image signal. ing.

The process until the subject light is stored in the image memory and read out will be described with reference to FIG. 8. In response to the vertical synchronizing signal V, a transfer pulse T is sent from the photoelectric conversion unit to the vertical transfer unit. The unnecessary charges are discharged by the unnecessary charge discharging pulse S. In the figure, unnecessary charges are discharged in period A, and charges are accumulated in period B to take in optical system information. In addition, transfer and writing to the memory are performed in the period C, reading from the memory is performed in the period D, and the zoom coefficient S is set at the timing of vertical synchronization. That is,
In this example, the optical system information obtained in the period B is delayed by one screen and read from the memory to set the coefficient.

FIG. 9 shows an example of a circuit for detecting optical system information by the microcomputer. Optical system 1 that changes depending on the zoom ratio
The position information of the zoom lens is provided with an external power source E1 connected in series with a variable resistor RV having a movable contact piece that responds to the position of the zoom lens, one end of which is grounded, and the resistor R1. The voltage at the connection point between the variable resistor RV and the resistor R1 whose resistance value changes according to is detected by the voltage detector 111 in the microcomputer 1 to which the timing signals V, T and S are supplied. The microcomputer 11 has a timing control circuit 114, and supplies a timing signal based on the signals V, T, S to each section. The voltage detection unit 111 obtains the moving distance of the zoom lens in the period B, the voltage / focal length conversion unit 112 obtains the focal length information, and then the parameter setting unit 113 sets the parameter at the timing of the timing signal V. Then, the coefficient S is output.

In addition to the above example, the optical distortion can be corrected at the time of writing the image information in the image memory. By doing so, the information of the accumulation period is used as it is in the writing period, so that more accurate distortion correction becomes possible.

Next, as an embodiment of the present invention, when performing optical distortion correction, the distortion is discretely distorted in a visually inconspicuous range based on the ratio of the distortion amount change to the change of the state of the optical system such as the focal length. An example will be described in which the memory capacity is reduced by obtaining the correction coefficient of.

As described above, since the optical distortion is defined by the focal length of the optical system and the distance from the optical axis of the image forming position, it is necessary to change the degree of correction if the focal length changes. However, it is generally said that human beings can detect optical distortion when the distortion ratio is 1% or more. Therefore, it is very complicated to change the correction state one by one according to the focal length, and the fluctuation of the correction due to the focal length detection error may occur.

Therefore, in this embodiment, the optical distortion correction is not changed within a range where there is no visual discomfort without setting the steps to be corrected more finely than necessary, that is, the optical distortion is predetermined. If it is within the range, the same correction coefficient is used to reduce the memory capacity and the amount of calculation required for optical distortion correction.

FIG. 10 shows the relationship between the focal length of the optical system and the amount of distortion. Regarding FIG. 10, focal length: 6.
Considering the range of 5 mm to 52 mm (8 times), the strain rate is in the range of -18% to + 2.5%, and in order to keep the corrected strain rate within ± 1%, {2.5-(-18)} / {1-(-1)} = 10.25 That is, the focal length may be detected in each of the division ranges divided into at least 11 stages. For example, as shown in FIG. 11, the distortion amount is in the range of + 1% to -1% (the focal length is 1 to 2).
Range), the distortion amount is in the range of -1% to -3% (the focal length is 2
.About.3), -3% to -5%, and when the focal length is within each divided range, the same coefficient is used for distortion correction. From FIG. 2, the amount of strain at the point of r = 100 is the maximum, so if the strain rate at r = 1 falls within the detection limit, the strain becomes even smaller in the range of r <1.

The operation processing procedure in the microcomputer of this embodiment will be described with reference to FIG. 12. The corresponding voltage is detected by the same means as the zoom lens position detecting means of FIG. 9 (step S1) and detected. The focal length is converted from the applied voltage (step S2), and the focal length zone is determined (step S3). This zone is a divided range of the focal length change range corresponding to the strain amount change range of ± 1%.
Then, it is determined whether or not the zone is changed (step S
4) If not, the process returns to step S1, and if there is a zone change, the coefficient S is calculated (step S5) and the coefficient is sent to the read address generation block.

Next, a description will be given of an example in which a screen break (a break in the angle of view) that occurs when electrically correcting the optical system distortion is removed by simultaneously performing the electronic zoom. As described above, the thread-winding type among the optical distortions is likely to occur during operation on the Tele side,
The barrel distortion is likely to occur when operating on the Wide side. Here, at the time of occurrence of the thread winding type distortion, the corner point A at the upper left part of the original image area (rectangular area) is inward as shown in FIG. 13 (B) by the conventional optical distortion correction as shown in FIG. 13 (A). Therefore, the shaded area in the figure is a signal from a portion without a photoelectric conversion element, and the angle of view with no image is cut off. Therefore, in this example, the electronic zoom is performed at the same time as the optical distortion correction to prevent the disconnection of the angle of view. The electronic zoom magnification does not have to be a large value, and may be set to a small value that does not cause a break in the angle of view.

The hardware configuration therefor is the same as that of FIG. 5, and a value of 0.9 is set as the pitch for setting the zoom magnification set in the H address pitch generation circuit 1202 and the V address pitch generation circuit 1204. Has been done. As the operation at this time, the microcomputer that has input the f value at that time calculates a correction amount for distortion correction based on the f value, and for the electronic zoom for preventing the angle of view cutoff corresponding to this. Both the pitches of H and V are output.

Another example is an automatic focus control (AF), automatic exposure control (A
E) and / or automatic white balance control (AWB) can be controlled, and the weighting coefficient given when the control is performed depends on the distortion correction amount for each divided area of the image area. This is an example in which basic information for control and each control are performed by using the video signal that has been changed or changed and distortion-corrected.

The following control example can be applied to each of the control objects of the AF control, the AE control and the AWB control, but the AF control will be described as an example. The first example is contrast information (basic control parameters) for each divided area.
By changing the weighting value for each area according to the amount of distortion, the effect of the error due to the distortion included in is obtained stably as the resulting weighted contrast information without distortion, and stable automatic Enables focus (AF) control. Further, if the image information after distortion correction is used, stable contrast information can be obtained regardless of distortion, and stable AF control can be performed. Therefore, stable AF control can be performed irrespective of the zoom ratio, image information without distortion can be obtained, and an optical system without a distortion correction lens can be used, so that the size and weight can be reduced.

In the second example, a sensor other than the image pickup device for obtaining a video signal is used as a sensor used for detecting contrast information, and the zoom which the light incident on the sensor forms an image on the image pickup device. The same effect can be obtained as long as it is of the TTL type that passes through the lens. In the second example, since the image information that has been electronically distortion-corrected is used to obtain the contrast information for each divided area, stable AF control is performed regardless of the zoom ratio without being affected by the distortion. By including the distortion correction inside the image processing device, it is possible to obtain image information without distortion, and similarly, since an optical system without a distortion correction lens can be used, it is possible to reduce the size and weight. . In the above description, not only the focus (AF) control but also the exposure (AE) control and the white balance (AWB) control can be performed in the same manner as the control target by using the brightness information and the color information as the control reference parameters.

The first example will be described with reference to FIG. In FIG. 14, components designated by the same reference numerals as those in FIG. 1 are components having the same function. The optical system 1 has
Zoom encoder 1 for detecting position information of the zoom lens
A, a zoom drive unit 1B for driving and controlling the zoom lens position
And a focus control unit 1C and an aperture drive unit 1D. The zoom lens position information from the zoom encoder 1A is supplied to the microcomputer 11, and the zoom control signal, the focus control signal and the aperture control signal are sent from the microcomputer 11 to the zoom drive unit 1B, the focus control unit 1C and the aperture control unit 1D.

As will be described later, the microcomputer 11 receives at least one signal of a contrast signal, a luminance signal and a color signal for each divided area, and appropriate correction data to be multiplied by each area information stored in the correction ROM 14. The (weighting data) is read and the correction data is sent to the read control circuit 12. Read control circuit 12
Sends a control signal to the image memory 5 and the interpolation circuit 6 based on the data stored in the correction ROM 15 through the above-described processing. The correction ROMs 14 and 15 include the microcomputer 11
It is not necessary to obtain necessary data by calculation in the or read control circuit 12.

The Y signal output from the image pickup process circuit 3 is directly supplied to the selector 17 after being converted into a contrast information signal by the filter 16 and supplied to the selector 17 as well. Selector 1
7 is a Y signal in response to a selection signal from the microcomputer 11,
Necessary information data is selected from the contrast signal and the C signal. The area integrator circuit 18 integrates the signal selected by the selector 17 for each divided area, and outputs the brightness signal and the color signal obtained from the area integrated signal to the microcomputer 11
As will be described later, focus control based on contrast information, exposure control based on luminance information, and white balance control processing based on color information (color difference signals such as RY signal and BY signal) are performed. .

15 (A) and 15 (B) show examples of weighting values for each divided area before the occurrence of yarn winding distortion and weighting values for each divided area before the occurrence of barrel distortion. . This weighting value shows an example of the calculated value originally set for AF, AE, AWB, etc. in consideration of the possibility that a main subject will be present in the approximate center of the screen. As is clear from the figure, when the thread-wound type distortion occurs, the image in the peripheral portion is distorted, so the correction for the ideal image must be performed correspondingly, and therefore the weighting value must be increased in the peripheral portion. There is.

FIG. 16 shows a change in weighting value when barrel distortion occurs and an example of correction thereof. In this example, the weighting value is set for each block including at least pixels, but a 4 × 4 block is treated as one divided area. Area A and area B are deformed by barrel distortion to become area C and area D, respectively. The weighting of each area when there is no distortion is as follows: Area A: {(0x3) + (1x3) + (2x4) + (3x3) + (4x2) + (5x1) } / 16 (=) 2 ... (a) Area B: {(7x1) + (8x2) + (9x3) + (10x4) + (11x3) + (12 × 2) +13} / 16 (=) 10 ... (b) The weighting of each area after the distortion is generated is area C: {(7 × 4) + (8 × 4) + (9 × 3)} /11(=)8...(c) Area D: {(11 * 5) + (12 * 3) + (13 * 2)} / 10 (=) 12 ... (d) It can be seen that the weighting coefficient has changed despite the same image component. Therefore, in order to perform the same weighting on the same image before and after the occurrence of distortion, a corrected weighting coefficient is obtained. That is, the weighting of each area after the correction is as follows: Area A originally has no image, and therefore Area A: (pixel data × number of pixels) / number of pixels = (0 × 16) / 16 = 0 area B: { ((A) × number of pixels of area C projected in area B) + ((b) × number of pixels of area D projected in area B) / (area projected in area B Total number of pixels of C and D) That is, = {(2 × 6) + (10 × 6)} / 12 = 6 Therefore, for area B, the correction coefficient may be set to 6/10. In this way, the correction of the weighting value is performed so that the average value of the products of the weighting value in the case where there is no distortion included in each divided area and the number of pixels of the small blocks is taken as the corrected weighting value.

The processing procedure of the example using the signal before distortion correction will be described with reference to FIG. 17. First, it is determined whether or not the automatic control mode is set (step S11), and it is determined that the automatic control mode is set. If so, the microcomputer 11 reads the zoom position information from the zoom encoder 1A (step S12) and codes the distortion correction amount (step S1).
3). Next, the correction coefficient of the weighting value of each area is read by referring to the table data of the distortion correction code (step S14), and the weighting data separately stored in the microcomputer is multiplied by the correction coefficient to calculate the weighting value of each area. Is corrected (step S15), the input of the area integrator 18 is switched by the selector 17 (step S16), and necessary control reference parameters (contrast information, brightness information, color information, etc.) are integrated by the integrator 18 ( Step S17), the area integration data is supplied to the microcomputer (read by the microcomputer 11) (step S1
8). Subsequently, the corrected weighting data is referred to (step S19), the integration data of each area is weighted (step S20), and new control data is calculated. Then, the control data is determined from the weighted area integration data (step S21). Then, the control data is output (step S22), and the process returns to step S11.

Next, as the second example, an example using a signal whose distortion has been corrected will be described. FIG. 18 is a block diagram of the configuration of this example, and the circuit parts denoted by the same reference numerals as those in FIG. 14 are circuit parts having the same function. In this example, the Y signal and the C signal whose distortion is corrected by the interpolation circuit 6 are used, the contrast information is obtained by the filter 16, and the selector 17 receives the distortion-corrected contrast signal, the Y signal,
The control reference parameter of the C signal is supplied to the microcomputer 11
Depending on the selection signal from the
Is supplied to. The area integrator 18 performs area integration on the input signal and supplies the integrated value to the microcomputer 11.

The processing procedure of this embodiment will be described with reference to FIG. First, it is determined whether or not it is the automatic control mode (step S31), and if it is the automatic control mode, the input of the area integrator 18 is switched and the necessary control reference parameter is output (step S32). Area integration is performed by the area integrator (step S33), and area integration data is calculated (step S34). Next, the weighting data is referred to (step S35), the integration data of each area is weighted (step S36),
A new control data calculation process is performed to obtain control data from the weighted area integration data (step S3).
7) After outputting the control data (step S38), the process returns to step S31.

In the present invention, it goes without saying that one or more of the above-mentioned embodiments and other examples can be used in combination, and various changes can be made without departing from the gist of the present invention. is there.

[0055]

As described above, the image processing apparatus according to the present invention has an optical distortion correction function with which there is little visual discomfort and the control operation is simplified.

[Brief description of drawings]

FIG. 1 is a configuration block diagram of a video processing apparatus related to the present invention.

FIG. 2 is a diagram showing an example of optical distortion characteristics of a zoom lens.

FIG. 3 is a diagram for explaining the operation of the device related to the present invention.

FIG. 4 is a diagram for explaining the operation of the device related to the present invention.

5 is an example of a circuit that realizes an electronic zoom function and an optical distortion correction function in the example shown in FIG.

FIG. 6 is a configuration block diagram of a video processing device in another example related to the present invention.

7 is an example of a circuit that realizes a camera shake correction function and an optical distortion correction function in the example shown in FIG.

FIG. 8 is a timing chart for explaining the operation of the video processing device related to the present invention.

9 is a configuration block diagram of a detection unit of zoom lens position information in the example shown in FIG.

FIG. 10 is a diagram showing the relationship between the focal length and the distortion amount of the optical system for explaining the embodiment of the present invention.

FIG. 11 is a diagram showing the relationship between the focal length and the distortion rate of the optical system for explaining the example of the present invention.

FIG. 12 is a diagram showing a configuration and an operation processing procedure of an embodiment of the present invention.

FIG. 13 is a diagram for explaining another example related to the present invention.

FIG. 14 is a configuration block diagram of another example related to the present invention.

15 is a diagram for explaining the example shown in FIG.

16 is a diagram for explaining the example shown in FIG.

17 is a flowchart showing an operation processing procedure of the example shown in FIG.

FIG. 18 is a block diagram of another configuration example based on the reference principle of the example shown in FIG.

19 is a flowchart showing an operation processing procedure of the example shown in FIG.

FIG. 20 is a diagram showing an example of optical system distortion.

FIG. 21 is a diagram for explaining correction of optical system distortion.

FIG. 22 is a configuration block diagram of a conventional video processing device.

[Explanation of symbols]

 1 Optical system 2 Image sensor 3 Imaging process circuit 4 A / D converter 5 Image memory 6 Interpolation circuit 7 D / A converter 8 TG circuit 9 SSG circuit 10 Write control circuit 11 Microcomputer 12A, 12 Read control circuit 13 Motion detector 14, 15, 19 Correction ROM 16 Filter 17 Selector 18 Area integrator

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Daikichi Shimohashi, Inventor Daikichi Shibaya, Tokyo 2-34-2 Hatagaya, Olympus Optical Co., Ltd. (72) Atsushi Inoue 2-43-2, Hatagaya, Shibuya-ku, Tokyo Olympus Optical Industry Co., Ltd.

Claims (1)

[Claims] 1. A conversion means for converting a video signal obtained by photoelectrically converting an image by an optical system into video data corresponding to the video signal, and writing or reading the video data by the conversion means into the storage means. An image processing apparatus comprising: a distortion correction unit that applies correction to the image data corresponding to an image distortion caused by the optical system, wherein the distortion correction unit includes an image distortion corresponding to a change in a focal length of the optical system. A video processing device, wherein correction data as data to be applied for each focal length section having a degree change within a predetermined range is given as data that is a basis of correction processing of the video signal. .