JP3152258B2 - Image processing apparatus and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to electronically enlarging an image.
The present invention relates to an image processing apparatus having an electronic zoom function and a method thereof .

[0002]

2. Description of the Related Art An image pickup apparatus having image pickup means for converting an optical image into an electric signal, for example, a video camera, an electronic still video camera, a television camera, etc., is provided with an image memory for storing a photographed image. A circuit configuration for applying video effects such as snap, solarization, camera shake correction, and electronic zoom has been proposed. It has been put to practical use in video tape recorders and camera-integrated video tape recorders. In particular, in a camera-integrated video tape recorder, an electronic zoom function has been attracting attention because it can also be used as an electronic camera shake correction function.

An image memory for temporarily storing photographed images usually requires a storage capacity for at least one screen (field or frame). For example, assuming that the luminance and chrominance signals for one field are both stored in 8 bits, 3 Mbits are required as it is, which not only increases the circuit size but also increases the cost. Therefore, a configuration has been proposed in which a compression encoding circuit is provided on the input side of the image memory and a decompression decoding circuit is provided on the output side of the image memory so that the storage capacity of the image memory can be reduced. For example, the captured image is compressed to 1/2 to 1/4 and stored in the image memory.

[0004]

When the electronic zoom function is incorporated in the circuit configuration for performing the compression / expansion processing before and after the storage in the image memory as described above, there are the following disadvantages. That is, in the electronic zoom, a part of an image is enlarged to a standard angle of view, and a missing pixel value is interpolated by a neighboring pixel value. As the magnification of the electronic zoom increases, the image quality degradation due to the interpolation processing becomes more conspicuous. On the other hand, as a compression method before an image signal is stored in an image memory, noise compression is usually employed in order to increase a compression ratio. In that case, the compression / decompression processing deteriorates the image quality to a considerable extent. When such a deteriorated image is electronically zoomed, the interpolation processing is performed based on the deteriorated image, so that the image quality is further deteriorated.

[0005] It is an object of the present invention to provide an image processing apparatus and method for solving such a problem.

[0006]

According to the present invention, there is provided an image processing apparatus comprising: input means for inputting image data; compression means for compressing image data input by the input means;
Storing image data processed by the compression means
Storage means for storing image data stored in the storage means;
Expansion means for expanding in accordance with the compression processing of the compression means
When the electronic zoom means for electrically enlarging the image data expanded by said expansion means, said electronic zoom means
Processing of the compression means and the expansion means in accordance with the zoom magnification.
Control means for controlling execution and non-execution of the physical operation . The image processing method according to the present invention further includes an input step of inputting image data,
A compression step of compressing image data, and processing in the compression step
Storing the stored image data in a storage medium;
The image data stored in the storage medium is compressed in the compression step.
An elongating step for elongating in response to a reduction process, and the elongating step
Electronic zoom to electrically enlarge image data expanded by
Step and according to the zoom magnification in the electronic zoom step.
Execution and non-execution of processing operations of the compression step and the decompression step
And a control step of controlling Ma
Further, the image processing apparatus according to the present invention is configured to input image data.
Input means, and image data input by the input means.
Compression means for compressing the data to 1 / n;
Compression processing by the compression means.
Selected image data, and
When selecting image data input from
First selection means for extracting a partial image, and the first selection
A memory for storing image data selected by the means,
Expansion means corresponding to the compression means, and electronic zoom means
The data read from the memory to the decompression means
And a second selectively supplied to one of the electronic zoom means.
Selecting means, wherein the magnification of the electronic zoom means is n times
When it is less than the above, by the compression means and the expansion means
The processed image data is electronically zoomed by the electronic zoom means.
When the magnification of the electronic zoom means is n times or more.
Is not processed by the compression means and decompression means
Electronically zooming image data with the electronic zooming means;
It is characterized by. Also, the image processing method according to the present invention
An input step of inputting image data, is input in the input step
A compression step of compressing the input image data to 1 / n;
Image data input in the compression process and compression in the compression process
Selecting the processed image data;
When selecting the image data input with
A first selecting step of extracting a minute image;
Storing the image data selected in the process in the memory
Decompressing the image data stored in the memory
Reading from the memory;
One of the expansion process and the electronic zoom process
A second selecting step of selectively supplying the electrons to the
When the zoom magnification in the zoom step is less than n times,
The image data processed in the compression step and the decompression step
Electronic zooming in the electronic zooming step;
When the zoom magnification at n is not less than n times,
Image data that has not been processed in the
It is characterized in that electronic zoom is performed in a zooming process.

[0007]

[0008]

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of the present invention. Reference numeral 10 denotes a photographing lens, 12 denotes an image sensor that converts an optical image into an electric signal, 14 denotes an A / D converter that converts an analog output of the image sensor 12 into a digital signal, 16
Is a digital signal processing circuit (DSP). DSP1
Reference numeral 6 performs well-known camera signal processing such as γ correction and color balance adjustment, and outputs an image signal in the form of an 8-bit luminance signal and a color difference signal, respectively. However, for the sake of simplicity, description will be made mainly on the luminance signal.

Reference numeral 18 denotes a compression circuit for compressing the output data of the DSP 16 to 1/4, 20 a selector for selecting the output of the DSP 18 or the output of the compression circuit 18, and 22 a compressed state of 1
An image memory capable of storing image data for a screen; 24, a decompression circuit corresponding to the compression circuit 18; 26, an image memory 22
Is supplied to the decompression circuit 24 or the output side of the decompression circuit 24.

Reference numeral 28 denotes an electronic zoom circuit for electronically zooming a signal from the expansion circuit 24 or the selector 26, reference numeral 30 denotes a D / A converter for converting a digital output of the electronic zoom circuit 28 into an analog signal, and reference numeral 32 denotes D / A conversion. Output terminal for outputting the output signal of the device 30 to the outside.

Although the details will be described later, the selector 20 has a function of selecting 8 inputs, a function of converting 8-bit data into two 4-bit data, and a horizontal and vertical designated range in the screen. Is provided with a gate function for passing image data. On the other hand, the selector 26 has a function of converting two 4-bit data into one 8-bit data in addition to a function of selecting two outputs.

Numeral 34 denotes a system control circuit for controlling the selectors 20 and 22 in conjunction with the zoom ratio of the electronic zoom circuit 28 in particular, and 36 denotes various instructions to the system control circuit 34 (for example, the zoom ratio of the electronic zoom circuit 28). , Etc.).

FIG. 2 shows a circuit configuration of the compression circuit 18. The input terminal 40 receives 8-bit data output from the DSP 16. The pre-filter 42 removes an oblique component from the luminance data from the input terminal 40 as a pre-processing of the sub-sampling. The sub-sampling circuit 44 thins the output of the pre-filter 42 in half in the horizontal direction,
The CM circuit 46 compresses the 8-bit data output from the sub-sampling circuit 44 to 4 bits by differential prediction coding. The output of the DPCM circuit 46 is applied from the output terminal 48 to the contact a of the selector 20 as the output of the compression circuit 18.

In the case of a luminance signal, the manner of compression will be specifically described. As shown in FIG. 3A, the luminance data of the original image is 640 pixels horizontally, 240 lines, and 8 bits.
It is assumed that the field image is about 1.23 Mbit. When this is sub-sampled by the sub-sampling circuit 44, the number of pixels in the horizontal direction is halved to 320, as shown in FIG. 3B, and is about 0.61 Mbit in one field. When this is further compressed from 8 bits to 4 bits by the DPCM circuit 46, the depth direction is halved as shown in FIG. 3C, and becomes about 0.3 Mbit.

FIG. 4 shows an internal circuit of the expansion circuit 24. The input terminal 50 has four bits from the a contact of the selector 26.
Enter the data. The DPCM decoding circuit 52 DPCM-decodes the 4-bit data from the input terminal 50,
Outputs bit data. The interpolation circuit 54 interpolates the pixels thinned out by the sub-sampling circuit 44. For example, insert a zero value. The average value of the pixel values on both sides or in the vicinity may be used. The output of the interpolation circuit 54 is smoothed by a post filter 56 and supplied from an output terminal 58 to the electronic zoom circuit 28.

That is, in the decompression circuit 24, the DPCM decoding circuit 52 changes the depth direction from 4 bits to 8 bits.
The image is expanded twice, and the interpolation circuit 54 doubles the number of pixels in the horizontal direction. As a result, the image data output from the output terminal 58 is 640 × 240 × 8 bits.

Needless to say, two circuits as shown in FIGS. 2 and 4 are provided in the compression circuit 18 and the expansion circuit 24 for luminance and color difference.

FIG. 5 shows the internal circuit of the selector 20, and FIG. 6 shows the internal circuit of the selector 26. The selector 20 is a DSP
16 is provided with a conversion function of dividing the 8-bit data output from 16 into upper 4 bits and lower 4 bits into two 4-bit data, and has a horizontal window signal H_WI
It has a gate function of passing image data of arbitrary horizontal and vertical regions in the screen by using the NDOW and the vertical window signal V_WINDOW. On the contrary, the selector 26 has a conversion function of allocating two 4-bit data to upper 4 bits and lower 4 bits of 8 bits. Details will be described later.

In this embodiment, the compression ratio of the compression circuit 18 is 1
/ 4, the zoom magnification of the electronic zoom circuit 28 is 4
When the value is twice or more, an effect peculiar to the present embodiment can be obtained.

First, the operation when the electronic zoom magnification of the electronic zoom circuit 28 is less than 4 will be described. At this time, the system control circuit 34 connects the selectors 20 and 26 to the a contacts, respectively. The image pickup device 12 converts an optical image by the photographing lens 10 into an electric signal, and the A / D converter 14
2 is converted to a digital signal. A / D converter 1
The output of 4 is processed by the DSP 16 and then applied to the compression circuit 18. The compression circuit 18 compresses the 8-bit image data to 1/4 as described above. The output of the compression circuit 18 is 4-bit data and is temporarily stored in the image memory 22 via the selector 20.

The compressed image data stored in the image memory 22 is read and applied to a decompression circuit 24 via a selector 26. The expansion circuit 24 expands the input compressed image data as described above, and applies it to the electronic zoom circuit 28.
The electronic zoom circuit 28 electronically enlarges the input image at a magnification specified by the system control circuit 34. That is, necessary pixel values are interpolated in the input image. For example, in the case of 2 × zoom, 453 horizontal pixels × 17 vertical pixels from the input image
0 lines are cut out and interpolated to output an image of 640 pixels × 240 lines.

Next, the operation when the electronic zoom magnification is four times will be described. In the case of 4 ×, as shown in FIG. 8, only the image data in the horizontal 320 pixels × 120 lines at the center of the original image need be stored in the image memory 22. That is, the data amount may be 1/4 of the image captured by the image sensor 12, and the amount is the same as that obtained by compressing the entire captured image to 1/4 by the compression circuit 18. Therefore, in the present embodiment, the data of the necessary central portion of the photographed image is stored in the image memory 22 without compression.

That is, the system control circuit 34 selects the selector 2
0 and 26 are respectively connected to the contact b. An image signal captured by the image sensor 12 is transmitted to the A / D converter 14 and the DSP 16
And applied to the b contact of the selector 20.

FIG. 9 shows an operation timing chart of the selector 20. Due to the H_WINDOW signal and the V_WINDOW signal, the 8-bit output (uncompressed data) of the DSP 16 is included in the horizontal line from the 161st pixel to the 480th pixel, and from the 61st line to the 180th line. Image data is selected. The selected 8-bit data is divided into lower 4 bits and upper 4 bits by the SET_HI signal and taken into the four-stage latch circuit, and 4 lower 4 bits and upper 4 bits are output in the order of 4 bits.
The data is sequentially output to the image memory 22 as bit data.

As described above, the image data of the central portion necessary for the electronic zoom is stored in the image memory 22 in an uncompressed state.

The uncompressed data stored in the image memory 22 is read and applied to the selector 26. Selector 2
FIG. 10 shows an operation timing chart of No. 6. By SEL,
The 4-bit data is taken into the upper 4 bits and lower 4 bits of the 8-stage latch circuit, and is converted into 8-bit data. When the eight bits are completed, the output is output to the electronic zoom circuit 28. The electronic zoom circuit 28 has a central 320
Image data of pixels × 120 lines is supplied. The electronic zoom circuit 28 performs an interpolation process on this, and 640 pixels ×
It outputs 240 lines of image data.

In the compression circuit 18 of the above embodiment, the compression is performed to 1/4 by the sub-sampling and the DPCM, but only the DPCM may be performed. In that case, the memory capacity required for the image memory 22 increases, but the pre-filter 42 and the post-filter 56 associated with sub-sampling and interpolation are not required, and the processing system is simplified.

[0030]

As can be easily understood from the above description, according to the present invention, even when the electronic zoom is performed at a predetermined level or more, the deterioration of the image quality is reduced, and the performance of the electronic zoom can be substantially improved.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is an internal circuit diagram of a compression circuit 18.

FIG. 3 is an explanatory diagram of compression of a compression circuit 18;

FIG. 4 is an internal circuit diagram of a decompression circuit 24.

FIG. 5 is an internal circuit diagram of the selector 20;

FIG. 6 is an internal circuit diagram of a selector 26.

FIG. 7 is an explanatory diagram of double electronic zoom.

FIG. 8 is an explanatory diagram of a 4 × electronic zoom.

FIG. 9 is a timing chart of the selector 20 at the time of 4 × electronic zoom.

FIG. 10 is a timing chart of the selector 26 at the time of 4 × electronic zoom.

[Description of Signs] 10: Photographing lens 12: Image sensor 14: A / D converter 16: Digital signal processing circuit (DSP) 18:
Compression circuit 20: Selector 22: Image memory 24: Decompression circuit 26: Selector 28: Electronic zoom circuit 30: D / A converter 32: Output terminal 40: Input terminal 42: Prefilter 44: Subsampling circuit 46: DPCM circuit
48: output terminal 50: input terminal 52: DPCM decoding circuit 54: interpolation circuit 56: post filter 5
8: Output terminal

Claims (4)

(57) [Claims] 1. An input unit for inputting image data, and the image data input by the input unit is compressed.
Compression means;Storing image data processed by the compression means
Storage means; The image data stored in the storage means is
Decompression means for decompressing in accordance with the compression process; Extended by the extension means Image dataToElectrically
Electronic zoom means for enlarging;Zoom magnificationAccording to the compression means
Execution and non-execution of the processing operation of the decompression meansControl
An image processing apparatus comprising: (2)An input step of inputting image data, A compression step of compressing the input image data, Store the image data processed in the compression step in a storage medium
Memory process to The image data stored in the storage medium in the compression step;
An elongation step for elongation corresponding to the compression processing; The image data expanded in the expansion step is electrically enlarged.
Electronic zoom process, The compression step according to the zoom magnification in the electronic zoom step.
And execution and non-execution of the processing operation of the decompression process
Image processing method comprising:
Law. (3)Input means for inputting image data; The image data input by the input means is reduced to 1 / n
Compression means for compressing; The image data input by the input means and the compression
Select image data that has been compressed by
Select the image data input by the input means.
Selecting means for extracting a partial image in a specified range when
When, Storing the image data selected by the first selecting means;
Memory and Decompression means corresponding to the compression means, Electronic zoom means; Decompressing the data read from the memory,
Of the electronic zoom means A second selection to selectively supply one
Selecting means, wherein the magnification of the electronic zoom means is less than n times
At the time of processing by the compression means and the decompression means
Electronic zoom of the obtained image data by the electronic zoom means.
When the magnification of the electronic zoom means is n times or more,
Images not processed by the compression means and decompression means
It is characterized in that data is electronically zoomed by the electronic zoom means.
An image processing device. (4)An input step of inputting image data, The image data input in the input step is compressed to 1 / n.
Compression process, Image data input in the input step and the compression step
Select the image data compressed by
Specified range when selecting image data
A first selection step of extracting a partial image of the box; The image data selected in the first selection step is stored in a memory.
A storage step for storing; Decompression step of decompressing image data stored in the memory
When, Electronic zoom process, Decompressing the data read from the memory;
A second selection which is selectively supplied to one of the electronic zoom steps;
And a zoom step in the electronic zoom step.
When it is less than n times, the compression step and the expansion step
The processed image data is electronically zoomed in the electronic zooming step.
The zoom magnification in the electronic zoom step is n times or more.
Occasionally, it is not treated in the compression step and the expansion step.
Electronic image in the electronic zoom step.
An image processing method characterized by the following.