JP3035992B2 - Video signal processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video signal processing device suitable for correcting an optical graphic distortion of a solid-state imaging device.

[Summary of the Invention] The present invention relates to a video signal processing apparatus for a solid-state imaging device, which uses a digital signal processing (sampling grid conversion processing) technique based on graphic distortion information of a lens to form each optical image having graphic distortion. By equivalently displacing the pixels in the same direction as the respective distortion vectors, or in such a manner that the respective distortion vectors have the same ratio, the respective pixels of the original distortion-free image are equivalently formed. The optical graphic distortion caused by the lens can be corrected relatively easily with a small circuit scale.

[Prior Art] As is well known, since an optical lens has aberration, for example, an optical image of a square lattice has a pincushion or barrel shape as shown in FIGS. 6A and 6B due to the aberration of the lens. Graphic distortion occurs. In the case of a zoom lens, as shown in FIG.
The direction of the distortion changes with a change in the focal length, and even with the same lens, the figure distortion changes from, for example, a barrel shape to a pincushion shape.

2. Description of the Related Art Conventionally, in an image pickup tube television camera, optical graphic distortion has been electrically corrected by controlling a deflection magnetic field or a current or a voltage for generating a deflection magnetic field.

[Problems to be Solved by the Invention] However, in a television camera using a solid-state imaging device such as a CCD, each pixel on the imaging device is aligned and fixed, and there is no movable portion such as an electron beam. ,
There is a problem that the graphic distortion cannot be corrected relatively easily as in the case of the above-mentioned image pickup tube camera.

However, since the figure distortion of the lens can be specified according to each use condition of the lens as shown in FIG. 7, in principle, for example, a frame memory is prepared,
It is conceivable to correct the graphic distortion on the memory by writing the video data from each pixel of the solid-state imaging device to the frame memory by shifting the address according to the graphic distortion for each lens use condition.

However, in this case, there is a problem that the scale of the specific circuit becomes unrealistically enormous.

In view of the foregoing, it is an object of the present invention to provide a video signal processing apparatus which has a small circuit scale and can relatively easily correct optical figure distortion.

[Means for Solving the Problems] A video signal processing apparatus according to the present invention is a video signal processing apparatus for processing a video signal from a solid-state imaging device in which an optical image of a subject is formed on a photosensitive surface via a lens. A two-dimensional interpolation operation circuit comprising an analog-to-digital converter for converting the video signal into a digital signal, a digital filter receiving the output of the analog-to-digital converter, an image memory receiving the output of the two-dimensional interpolation operation circuit, And a correction amount memory for providing a coefficient for interpolation to the two-dimensional interpolation calculating circuit based on the distortion amount of the lens. The sampling lattice conversion unit determines the distortion amount of the lens. The amount of displacement is controlled based on the information shown, and when the amount of displacement of the sampling grid at the time of conversion is a displacement of an integer number of pixels of the imaging device, the image memo is stored. When the displacement is smaller than one pixel, it is controlled by the interpolation processing of the digital filter.

In a video camera for processing a video signal from a solid-state imaging device in which an optical image of a subject is formed on a photosensitive surface via a lens, an analog-to-digital conversion for converting the video signal into a digital signal is provided. Two-dimensional interpolation operation circuit comprising a device and a digital filter receiving the output of the analog-digital converter, an image memory receiving the output of the two-dimensional interpolation operation circuit, and the two-dimensional interpolation operation circuit based on the distortion amount of the lens And a correction amount lens for giving a coefficient for interpolation to the lens. The sampling grid conversion means controls the amount of displacement based on information indicating the amount of distortion of the lens, and controls the displacement. When the amount of displacement of the sampling grid at the time of conversion is the displacement of the integer number of pixels of the image sensor, controls the address of the image memory, When the pixels smaller displacement is controlled by the interpolation process of the digital filter, and corrects the video signal from the solid-state imaging device.

[Operation] According to such a configuration, optical figure distortion is corrected relatively easily with a small circuit scale.

[Embodiment] An embodiment of a video signal processing apparatus according to the present invention will be described below with reference to FIGS.

FIG. 1 shows the configuration of an embodiment of the present invention, and FIGS. 2 and 3 show the configuration of the main part thereof.

In FIG. 1, (11) is a zoom lens, and (12) is a solid-state image sensor such as a CCD, for example, and an optical image of a subject (11) is transferred to the image sensor (1) through the zoom lens (11).
It is tied on the photosensitive surface of 2).

(20) is a sampling grid conversion means, which is a two-dimensional digital filter (21) for interpolation calculation and an image data memory (2).
2), the output of the image sensor (12) is supplied to a digital filter (21) through an AD converter (23) and a buffer memory (24), and the output of the digital filter (21) is The data is written to the image data memory (22). An address signal generating circuit (25) includes a counter (25c). Writing and reading of the image data memory (22) are controlled by an address signal from the signal generating circuit (25). The image data read from the memory (22) is led to a terminal OUT via a DA converter (26).

(27) is a system control circuit, which is connected to the correction amount memory (28) and supplies lens information from the zoom lens (11). The memory (28) stores a correction amount (vector) predetermined for each part of the screen for each lens use condition, and a control signal based on the correction amount is transmitted from the control circuit (27) to the two-dimensional digital filter (21), The signals are supplied to the buffer memory (24) and the address signal generation circuit (25), respectively.

Generally, in the interpolation operation, it is ideal to use an interpolation function of the form y = (sin x) / x. However, when a cubic convolution function of the form y = ax ³ + bx ² + cx + d is used, two-dimensional digital The filter can be configured on the scale of a 4 tap × 4 tap FIR filter.

When calculating an m-tap × n-tap FIR filter, the two-dimensional digital filter is configured as shown in FIG. 2. When calculating a 2-tap × 2-tap FIR filter with a minimum scale, FIG. It is configured as shown in FIG.

In FIG. 2, reference numeral (30) denotes a two-dimensional digital filter, which comprises a line delay line (31) having a delay time of one line period (H) and a product-sum operation circuit (41);
The input terminal (30i) has m-1 line delay lines (31a),
(31b) ‥‥ (31m−1) are cascaded in reverse order, and m arithmetic circuits (41
a), (41b) ‥‥ (41m) are connected respectively. Each of the product-sum operation circuits (41a) to (41m) has a finite impulse response (FIR) type n-order transversal filter configuration.

That is, in the first product-sum operation circuit (41a), n-1 unit delay units (42a), (42b) ‥‥ (42n-1) each having a delay time of one data period are cascaded in reverse order. Then, n multipliers (43a) to (43n) are connected to the output terminal, each connection midpoint, and the input terminal, respectively. Multiplier (43
a) to (43n) have coefficient memories (44a) to (44n) respectively
Are connected, and the outputs of the multipliers (43a) to (43n) are connected to the adder (45). The second to (m-1) th product-sum operation circuits (41b) to (41m-1) are similarly configured. The m-th product-sum operation circuit (41m) has the same configuration as that of the first product-sum operation circuit (41a), but for convenience of explanation, “4” is added to the sign of each corresponding part, and the unit delay unit, Multipliers, coefficient memories, and adders are (46), (47), (48), and (49), respectively.
And

The coefficient memory (44) of each product-sum operation circuit (41a) to (41m)
a) to (48n) are supplied with control signals from the system control circuit (27) via control terminals (30c), and the outputs of the product-sum operation circuits (41a) to (41m) are added to adders ( It is led to the output terminal (30o) via 49).

When the configuration of the two-dimensional digital filter is minimized, the configuration shown in FIG. 3 is obtained. However, since the configuration is basically the same, the same components as those shown in FIG. And a duplicate description is omitted.

Next, the basic operation of one embodiment of the present invention will be described with reference to FIGS.

The number of pixels of the image pickup device (12) is enormous, for example, 720 × 525, but for simplicity, in FIG. 4, both the vertical and horizontal sampling grids are represented by five sampling grids. Vertical line
The horizontal line is made to intersect the optical axis of the lens.

When the figure distortion by the lens is in the positive direction, the optical image (2) of the square lattice (subject) (1) becomes a pincushion shape as shown by a solid line in FIG. Tied on the plane (12s). In this case, each grid point Pij of the optical image (2) is located outside of the corresponding grid point Qij of the distortion-free image (3) (see FIG. B) as shown by a thin line. The distortion of each grid point Pij is represented by a vector Dij.

In this embodiment, each pixel represented by the lattice point Pij of the distorted optical image (2) is equivalently displaced in a direction substantially the same as the distortion vector Dij by using a digital signal processing technique. An original distortion-free image (3) represented by the corresponding lattice point Qij is equivalently formed. That is, by performing the conversion processing of the sampling grid, the figure distortion due to the lens is corrected.

From another point of view, this graphic distortion correction is performed by, as shown by a thin line in FIG. 4C, pixels arranged in a horizontal direction on the photosensitive surface (12s) of the image sensor, for example, by a distortion correction vector Cij.
And a barrel-shaped photosensitive surface is equivalently formed.

The pixels of the solid-state imaging device (12) are arranged on the photosensitive surface (12s) indicated by a chain line in FIG. 4A, and video data from each pixel is sequentially output by horizontal scanning.

On the other hand, the image data memory (22) is given an address corresponding to the pixel position of the original image, that is, the pixel position of the undistorted image (3).
It is written to the address shifted by the distortion correction vector Cij in FIG. As a result, on the data memory (22), an image with figure distortion corrected can be obtained. And the memory (2
This image data is read out in ascending order of the address in 2), and an original distortion-free image can be obtained.

As described above, simply writing addresses to the image memory while shifting the addresses causes unevenness in pixels due to duplication or omission, thereby deteriorating the micro linearity of the reproduced image.
In this embodiment, displacement of an integer number of pixels of the image sensor (12) is controlled by address processing of an image memory (22). For displacements smaller than one pixel, a correction amount stored in a memory (28) is used. Is controlled by the interpolation operation of the digital filter (21).

When the optical image (2) is reduced, the sampling frequency is increased. Conversely, when the optical image (2) is enlarged, the sampling frequency is decreased.

In the case of the conversion for increasing the sampling frequency, pixel data of a group of sample points P of a group of widely-spaced sampling grids X and Y before conversion shown by a solid line in FIG. As a result, the pixel data of one sample point Q at the center of the converted narrow sampling grids U and V shown by the broken line in FIG.

Note that the wide-spaced sampling grids Xi, X shown by solid lines in FIG.
j, Xk ‥‥; Yi, Yj, Yk ‥‥, within a region surrounded by four sampling points P at the vertices of a square, narrow sampling grids Ui, Uj, Uk, U1 indicated by broken lines There may be a plurality of sample points Q of {; Vi, Vj, Vk, V1}. In this case, the subsequent image data from the A / D converter (23) is stored in the buffer memory (24).
And a plurality of interpolation calculations are performed.

In addition, as is apparent from FIG. 7 described above, in this embodiment, the ratio of enlargement / reduction is at most 10%.
As shown in the drawing, the sampling grids before and after the conversion can be approximated as being locally orthogonal at equal intervals, when viewed locally.

Specifically, an interpolation operation is performed in a two-dimensional digital filter (30) as shown in FIG. 2 or FIG.

That is, in FIG. 2, the image data from the input terminal (30i) is directly supplied to the m-th product-sum operation circuit (41m), and at the same time, the line delay lines (31m-1) to (31a) are connected. Via the product-sum operation circuits (41m-1) to (41a). Thereby, at an appropriate time, the data of the n pixels of the i-th line of the image sensor (12) is input to each of the multipliers (43a) to (43n) of the first arithmetic circuit (41a). The data of the pixel in the (i + 1) th line is input to the second arithmetic circuit (41b), and the (m−1) th arithmetic circuit (41
The data of the pixel of the (i + m) -th line is input to m-1), and the multiplier (47a) of the m-th arithmetic circuit (41m)
Data of n pixels on the (i + m-1) th line are input to .about. (47n).

Each memory (44a) of the product-sum operation circuits (41a) to (41m)
(48n) stores weighting coefficients corresponding to the spread of the interpolation function, and the corresponding multipliers (43a)
In (47n), the coefficient is multiplied by the pixel data of each sample point to give a predetermined weight. The outputs of the multipliers (43a) to (47n) are added in the adder (49) to form the pixel data of the sample point Q after the conversion, which is derived from the terminal (30o).

The weighting coefficients stored in each of the memories (44a) to (48n) differ depending on the relative positions of the sample points Q after conversion and the four sample points P surrounding them before conversion, but in this embodiment, For example, assuming a 1/8 subdivision grid, the coefficients for each subdivision grid point are stored in the memories (44a) to (48n), and the coefficients of the subdivision grid points closest to the converted sample point Q are stored in the memory (44a). 44a) to (48n).

This interpolation calculation is sequentially performed along each pixel line horizontally arranged on the photosensitive surface (12s) of the image sensor as shown in FIG. 4C, and the calculation result is as described above. This is data on the tip position of the distortion correction vector Cij in FIG.

Therefore, in this embodiment, when writing the output of the two-dimensional digital filter (21) to the image data memory (22), a required write address signal corresponding to the current interpolation processing pixel is output from the address signal generation circuit (25). Output
Interpolated pixel data is written to a predetermined address on the memory (22).

In this case, the image data interpolated for one horizontal line on the photosensitive surface (12s) of the image sensor is stored in the memory (22).
The required capacity of the image data memory (22) is determined by how much the data is written in the vertical direction. In this embodiment, a large capacity such as a frame memory or a field memory is unnecessary, and a capacity of about the number of horizontal lines corresponding to the vertical component of the maximum distortion vector is sufficient.

As described above, the correction amount stored in the memory (28) is
Although determined for each part of the screen, the same value is used for each quadrant of the screen due to the point symmetry of the lens, thereby saving the capacity of the memory (28).

In addition, as is apparent from FIG. 7 described above, in this embodiment, since the ratio of the optical figure distortion is at most 10%, the difference in the correction amount between adjacent pixels of the image sensor (12) is small. It is. By storing only this difference, the capacity of the memory (28) can be saved.

In the above-described embodiment, each pixel of the distorted optical image is displaced in the same substantially opposite direction as the respective distortion vector.However, each pixel is identical to the distortion vector so that each distortion vector has the same ratio. The displacement can be further controlled in the direction, and the displacement can be controlled so that each pixel after the displacement is aligned on a straight line.

[Effects of the Invention] As described above in detail, according to the present invention, each pixel of an optical image having a graphic distortion is obtained by using a digital signal processing (sampling grid conversion processing) technique based on the graphic distortion information of a lens. Is displaced equivalently in the same direction as the respective distortion vectors or so that the respective distortion vectors have the same ratio, so that the original distortion-free image is equivalently formed for each pixel. Therefore, in the solid-state imaging device, a video signal processing device that can relatively easily correct optical graphic distortion due to a lens with a small circuit scale can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of a video signal processing apparatus according to the present invention, and FIGS. 2 and 3 are block diagrams showing the configuration of the main parts of an embodiment of the present invention. 5 is a schematic diagram for explaining the operation of one embodiment of the present invention, FIG. 6 is a schematic diagram for explaining the present invention, and FIG. 7 is a diagram of a lens for explaining the present invention. FIG. 4 is a diagram illustrating distortion characteristics. (11) is a zoom lens, (12) is a solid-state image sensor, (12)
s) is the photosensitive surface, (20) is the sampling grid conversion means, (21),
(30) is a two-dimensional digital filter (interpolation operation circuit),
(22) is an image data memory, (25) is an address signal generation circuit, (27) is a system control circuit, (28) is a correction amount memory, (31) is a line delay line, and (41) is a product-sum operation circuit. is there.

Claims (2)

(57) [Claims] 1. A video signal processing device for processing a video signal from a solid-state imaging device in which an optical image of a subject is formed on a photosensitive surface via a lens, comprising: an analog-digital converter for converting the video signal into a digital signal. A two-dimensional interpolation calculation circuit comprising a converter and a digital filter receiving the output of the analog-digital converter; an image memory receiving the output of the two-dimensional interpolation calculation circuit; and the two-dimensional interpolation calculation based on the amount of distortion of the lens And a correction amount memory for providing a coefficient for interpolation to the circuit. The correction means controls the amount of displacement based on the information indicating the amount of distortion of the lens. When the displacement amount of the sampling grid at the time is a displacement of an integer number of pixels of the image sensor, the address of the image memory is controlled, and
A video signal processing device, wherein when the displacement amount is smaller than the number of pixels, the displacement is controlled by interpolation processing of the digital filter. 2. A video camera for processing a video signal from a solid-state imaging device in which an optical image of a subject is formed on a photosensitive surface via a lens, comprising: an analog-digital converter for converting the video signal into a digital signal; A two-dimensional interpolation operation circuit comprising a digital filter receiving an output of a digital converter; an image memory receiving an output of the two-dimensional interpolation operation circuit; and an interpolation circuit for the two-dimensional interpolation operation circuit based on the amount of distortion of the lens. And a correction amount lens that gives a coefficient of the following. The sampling grid conversion means controls the displacement amount based on information indicating the distortion amount of the lens, and the control is performed at the time of conversion. When the displacement amount of the sampling grid is a displacement corresponding to an integer pixel of the image sensor, the address of the image memory is controlled, and the displacement is smaller than one pixel. A video camera wherein the displacement amount is controlled by interpolation processing of the digital filter to correct a video signal from the solid-state imaging device.