JP2009124278A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device (video camera for consumer use), capable of performing advanced signal processing using an image memory, and of being reduced in size with suppressed power consumption, and operating at a low-power consumption. <P>SOLUTION: The imaging device is equipped with a camera signal processing circuit, having a data compression and decompression means 105, and when image data to be applied to image processing for high image quality are temporarily stored in an image memory, the image data are stored in the memory, after compressing the image data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a small video camera with high image quality.

  As environments surrounding consumer video cameras, personal computers and high-definition televisions capable of displaying high-definition images are becoming widespread. Therefore, in the development of consumer video cameras, it has become an important issue to provide users with products that can shoot with higher resolution and higher image quality than before.

  To realize a high-resolution and high-quality video camera, we developed moving image processing technology for image quality improvement using an image memory and high-resolution still image shooting processing technology using an image sensor with a large number of pixels. It will be necessary to install it in the product. For example, according to Patent Document 1, the presence / absence of motion is detected from a plurality of interlaced images that are temporally continuous, and interpolation in the screen using past field information and interpolation in the time axis direction are performed. Are described in terms of techniques for generating good progressive images by adaptively switching between. By using the image memory in this way, it is expected that high-quality image generation can be performed by performing advanced filtering processing using information in the time axis direction.

JP-A-62-111586

  On the other hand, a consumer video camera generally has many opportunities to shoot with a hand, so that a small and lightweight camera is eagerly desired by users. Low power consumption is an important elemental technology for making the device small and light. The reason for this is that when the power consumption increases, it is necessary to increase the battery capacity to cope with the decrease in continuous use time, or it is necessary to take heat measures to release the heat generated by power consumption to the outside. This is because the size increases. In this way, in a consumer video camera, in order to combine a small size and light weight with high image quality and high performance, a technique for performing advanced and complicated signal processing with as little power consumption as possible is important.

  An object of the present invention is to solve a technical problem for reducing power consumption in order to reduce power consumption while performing advanced signal processing using an image memory in a consumer video camera.

  In the present invention, the camera signal processing circuit is provided with data compression / decompression means so that the image data is compressed when temporarily storing the image data for image processing for improving the image quality in the image memory. Then store it in memory.

  In the video camera of the present invention, since data is compressed by data compression / decompression means provided in the camera signal processing circuit and then written to the image memory, power consumption is reduced by suppressing the frequency of access to the memory. . In addition, since the data is compressed and stored in the memory, the amount of image data can also be reduced as compared with a method without compression means. That is, the quantity of memory itself can be reduced, which also leads to a reduction in power consumption.

  In the present invention, in a video camera having an image signal processing circuit for performing high image quality processing using frame correlation, the image signal processing circuit is provided with a means for compressing / decompressing a digital data sequence, and is used as a reference for frame correlation processing. A video camera system with low power consumption that suppresses the frequency of access to the memory as much as possible by using the compression / decompression means to store the compressed data in the memory when reading and writing to the image memory. In order to achieve this, it takes the following form.

FIG. 1 shows an example of a video camera system according to the present invention. In the figure, 101 is an imaging lens, 102 is an imaging device, 103 is a Y / C conversion circuit, 104 is a high image quality processing circuit, 105 is a microprocessor that controls a camera signal processing open circuit, 106 is a compression / decompression circuit, and 107 is A memory interface circuit 108 is a frame memory. Reference numeral 100 denotes a camera signal processing circuit including a Y / C conversion circuit 103, a high-quality processing circuit 104, a microprocessor 105, a compression / decompression circuit 106, and a memory interface circuit 107. These are often integrated into an LSI form. The general operation of this video camera system is as follows. Light arriving from the subject is imaged on the light receiving surface of the image sensor 102 by the imaging lens 101. The image sensor is a general image sensor such as a CMOS (Complimentary Metal Oxide Semiconductor) sensor or a CCD (Charge-Coupled Device) sensor. Photodiodes are arranged in a grid on the light receiving surface of the image sensor. Yes. Each photodiode sequentially converts the reaching light into electric energy and stores it as electric charges. In addition, the electric charge stored in the image pickup device is repeatedly discharged as an unnecessary charge or read out as an image pickup signal every arbitrary time. In the image sensor, the exposure time can be optimized by controlling the interval between discharge and readout. The image pickup signal read from the image pickup device is supplied to the camera signal processing circuit 100, and is converted from the RGB signal, which is the image pickup signal, into luminance and color signals by the Y / C conversion circuit 103. FIG. 2 is a diagram illustrating an image when an interlace signal of Y: Cr: Cb (4: 2: 2) is generated from an image signal that is progressively read out from an image sensor having an RGB Bayer array color filter. . In the figure, the upper part shows the luminance signal and the color signal in the first field, and the lower part shows how the luminance signal and the color signal are generated in the second field. For example, YCrCb signals corresponding to the coordinates of B22 of the RGB imaging signal are Y22, Cb22, and Cr22. That is, the luminance signal and the color signal at each coordinate when the vertical coordinate is v and the horizontal coordinate is h are shown as Yvh, Cbvh, and Crvh in FIG. Then, the signal of the coordinate of the line which interpolates mutually in the perpendicular direction of the signal produced | generated by the 1st field and the signal produced | generated by the 2nd field. Further, the generated color signal has only even coordinates with respect to the horizontal coordinate, because this is an example in which an image signal in 4: 2: 2 format is handled. In general, image signals in 4: 2: 2 format reduce the amount of information by thinning out in the horizontal direction using the low visibility of the resolution with respect to color. Luminance signals and color signals for specific coordinates can be generated by first predicting R, G, and B signals for the specific coordinates, and then multiplying the predicted RGB values by coefficients. As the simplest method, there is a method of generating the luminance signal Y and the color signals Cb and Cr by predicting the R, G and B values of each coordinate from the region of three vertical pixels and three horizontal pixels. For example, the luminance signal and the color signal for the coordinates of B22 can be generated by the following formula using the simple prediction method.
R22 = (R11 + R13 + R31 + R33) ÷ 4 (Formula 1)
B22 = B22 (Expression 2)
G22 = (G12 + G21 + G23 + G32) ÷ 4 (Expression 3)
Y22 = + 0.299 × R22 + 0.587 × G22 + 0.114 × B22
Cb22 = −0.196 × R22−0.331 × G22 + 0.500 × B22
Cr22 = + 0.500 × R22−0.419 × G22−0.081 × B22
As described above, the Y / C conversion circuit generates the luminance signal and the color signal. Here, in order to simplify the description, a process of generating a luminance signal and a color signal from an area of 3 pixels × 3 pixels is illustrated as an example, but in reality, RGB signal interpolation processing (Equations 1 to 3) is performed. In order to make the band of each signal uniform, it is common to apply filtering to a wide area.

The luminance signal and the color signal generated by the Y / C conversion circuit 103 are sent to the high image quality processing circuit 104 and the compression / decompression circuit 106. In the compression / decompression circuit, the luminance signal and the color signal generated by the Y / C conversion circuit are compressed. Code data indicating luminance and color generated by the compression processing is temporarily stored in the frame memory 108 via the memory interface circuit 107. Further, the compression / decompression circuit reads out the code data temporarily stored in the frame memory via the memory interface, returns it to the baseband luminance signal and color signal, and supplies them to the image quality processing circuit. FIG. 3 shows an example of the configuration of the compression / decompression circuit. In the figure, 201 denotes a DPCM (Differential Pulse Code Modulation) processing circuit, 202 denotes a compression processing circuit, 203 denotes a write buffer, 204 denotes an expansion processing circuit, 205 denotes an inverse DPCM processing circuit, and 206 denotes a read buffer. The DPCM processing circuit performs DPCM processing individually for each signal component of the luminance signal Y, the color signal Cb, and the color signal Cr (see FIG. 4). As a DPCM processing procedure, first, the head image signal level of each horizontal line is first output as a reference value, and thereafter a difference value from adjacent data is output. FIG. 5 shows the state of DPCM processing when attention is paid to a certain line. The upper stage corresponds to the signal before DPCM processing, and the lower stage corresponds to the signal after DPCM processing. For example, before the DPCM process Y (luminance) signal Yi, Cbj and Cb signal, when the Crk the Cr signal, after the DPCM processing Y signals D _i, a Cb signal Db _j, a Cr signal and Dr _k corresponding I am letting. At this time, the state of the DPCM processing is expressed by the following formula using the reference numerals shown in FIG.

Di = {D0, D1, D2,..., (Dn-1)}
= {Y0, Y0-Y1, Y1-Y2, ..., (Yn-2)-(Yn-1)
Dbj = {Db0, Db1, Db2,..., (Dbn-1)}
= {Cb0, Cb0-Cb1, Cb1-Cb2, ..., (Cbm-2)-(Cbm)}
Drk = {Dr0, Dr1, Dr2,..., (Drn-1)}
= {Cr0, Cr0-Cr1, Cr1-Cr2, ..., (Crm-2)-(Crm)}
In the DPCM processing circuit, the head of data is determined by using an assert signal indicating the valid range of data. For example, it takes the form shown by W_REQ in FIG.
A compression processing circuit 202 performs data compression by a lossless compression method and transfers data to the write buffer 203. An example of the lossless compression algorithm used here is a variable-length code method based on the Huffman method. In the Huffman method, it is generally known that the amount of data can be compressed by encoding so as to give a short code length to a value having a high appearance probability. On the other hand, a natural image signal generally has a property that the correlation between adjacent image signals is high, that is, a property in which the signal levels of adjacent luminance and color are close to each other in most cases. The smaller the absolute value of the processed image signal, the higher the probability of occurrence. By utilizing this property, it is possible to realize lossless data compression by applying Huffman coding that gives a code having a shorter code length to a lower absolute value having a higher appearance probability. The code data compressed by the compression processing circuit is supplied to the write buffer 203 as a bit string, and the bit buffer is stored in a FIFO (First In First Out) buffer while packing the bit string in units of bytes. The accumulated byte data is transferred to the memory interface 107 in burst units for accessing the frame memory 108. In addition, when the compression processing for one line is completed as an exception processing in the write buffer, if the data amount is less than the burst unit, dummy data is embedded and data transfer to the memory interface 107 is also performed.

  On the other hand, in reading data from the frame memory 108, it is necessary to read data based on a read request signal (R-REQ) as shown in FIG. 6, and the latency at that time, that is, from request generation to read data output. Therefore, the compressed data is read from the frame memory 108 to the decompression processing circuit 204 before the request is generated. The read compressed data is converted from byte-unit data to serial data by the decompression processing circuit, and is combined with the DPCM signal. The composite DPCM signal is subjected to inverse DPCM conversion processing by the inverse DPCM processing circuit 205 and stored in the read buffer 206 as prefetched data. The read buffer is a FIFO buffer, and transfers image data of past frames to the high image quality processing circuit 104 in response to a read request.

  The high image quality processing circuit uses a past field luminance signal and color signal temporarily stored in the frame memory 108 and a current luminance signal and color signal generated by the Y / C conversion circuit to generate a high quality image. Perform processing. An example of the image processing is, for example, dot number conversion of an image signal. The dot number conversion is a resolution conversion process that originally expresses an image of n dots in the vertical direction and m dots in the horizontal direction with a quadruple number of dots or a dot number of 1/4. Usually, the resolution conversion processing is generally performed by predicting and storing the signal level of the coordinate after conversion from the surrounding signal level. FIG. 7 shows an example of the vertical interpolation in the case of doubling the number of dots in the vertical direction of the interlace signal. The upper row shows the interpolation of the first field, and the lower row shows the second interpolation. This shows the state of interpolation of the field. Here, an interpolation state is illustrated assuming that an image offset in the vertical direction by 0.5 dots is created in the first field, and an image that is not offset at all from the original image is created in the second field. When doubling the number of vertical dots in an interlaced image, according to this interpolation method, the past field information can be used to store the interpolated image from dots with close coordinates. The stored data can be predicted with higher accuracy than the interpolation method (FIG. 8) that does not use field information. As for the high-quality processing, the case where the number of dots is converted has been described as an example. However, the high-quality processing is not limited to this, and the image quality using inter-frame information such as noise reduction processing by inter-frame prediction is used. It refers to all the processing that improves.

  In the first embodiment, in a camera system that uses past field information for high image quality processing, past field information referred to in high image quality processing is temporarily stored in a frame memory via lossless compression / decompression. As described above, an embodiment in the case of temporarily storing in the frame memory after performing lossy compression processing will be described. FIG. 9 and FIG. 10 are diagrams showing an example when irreversible compression processing is applied to the compression / decompression circuit 106 in FIG. In FIG. 9, reference numerals 301 and 302 denote bit shift circuits. The bit shift circuit 301 performs bit shift in the direction of reducing the bus width, and the 302 bit shift circuit is based on the bus width reduced by the 301 bit shift circuit. Bit shift in the return direction. On the other hand, FIG. 10 shows a low-pass filter circuit 401, which plays a role of band-limiting the data stored in the field memory. In any case, irreversible compression is applied by the newly added circuit. However, since the compression effect can be expected more than in the first embodiment, the frequency of access to the frame memory can be further reduced. In this case, the high-quality image processing using the past field information may conceivably use the past information indirectly. In the method of using indirectly, the compression / decompression process does not necessarily need to be reversible. As an example of such high image quality processing, for example, when filtering processing is performed on a signal generated by the Y / C conversion circuit 103, the filtering coefficient is changed in accordance with past field information within the same screen. Available in case. As a parameter for changing the filtering coefficient, for example, there is a motion vector, and as a filtering process, a band limiting process is raised. In a digital recording camera, an image compression format such as MPEG (Moving Picture Experts Group) is generally used, and the compression efficiency is closely related to the frequency distribution of the baseband image signal to be compressed. For example, if the compression efficiency is forcibly increased for a signal having many high-frequency components, the compression processing fails and causes block noise or the like. On the other hand, in human visual characteristics, since the frequency resolution for a fast moving object is lowered, even if the signal is band-limited at the baseband stage, a large amount cannot be seen. Therefore, the high-quality image processing circuit can detect only motion information from past field information, and it is possible to limit the band severely in a portion where motion is severe and loosely restrict the bandwidth to a portion where there is little motion. Since image processing suitable for the image compression format is realized, a high-efficiency and high-quality moving image compression signal can be generated. As described above, in the second embodiment, by adopting a configuration that enables irreversible compression processing, reversible and irreversible are appropriately selected in the compression processing of image data stored in the image memory for high-quality processing as necessary. By switching, it becomes possible to perform high-quality image processing more efficiently.

  The preferred embodiments of the present invention have been described above. As an embodiment, the present invention can be used in a video camera that performs high-quality image processing using an image memory.

An example of the video camera system in this invention. Example 1 The image figure of interlace signal generation. Example 1 FIG. 3 shows an example of the configuration of the compression / decompression circuit. Example 1 Explanatory drawing of making color signal Cb and color signal Cr into separate DPCM. Example 1 State of DPCM processing when paying attention to one line. Example 1 FIG. 6 is an explanatory diagram of data read timing from the frame memory. Example 1 An example of a state of vertical interpolation. Example 1 An interpolation method that does not use past field information. Example 1 An example when irreversible compression processing is applied to a compression / decompression circuit. (Example 2) An example when irreversible compression processing is applied to a compression / decompression circuit. (Example 2)

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Camera signal processing circuit 101 Image pick-up lens 102 Image pick-up element 103 Y / C conversion circuit 104 High image quality processing circuit 105 Microprocessor 106 Compression / decompression circuit 107 Memory interface circuit 108 Frame memory 201 DPCM (Differential Pulse Code Modulation) processing circuit 202 Compression processing circuit 203 Write Buffer 204 Decompression Processing Circuit 205 Inverse DPCM Processing Circuit 206 Read Buffer 301 Bit Shift Circuit 302 Bit Shift Circuit 401 Low Pass Filter Circuit

Claims (4)

  An image sensor that converts light into an electrical signal, an optical lens that forms an optical image of the subject on the image sensor, and an image signal that indicates the brightness and color of the subject from the image signal read from the image sensor An image pickup apparatus configured by a signal processing circuit that is generated by the conversion processing circuit that converts the image pickup signal into an image signal of a signal format determined by a certain standard, and the conversion processing circuit A memory interface circuit for temporarily storing an image signal as image information in a memory means, a processing circuit for generating a new image signal using a plurality of pieces of image information and image signals temporarily stored in the memory means, A compression / decompression circuit that compresses and decompresses the image signal generated by the conversion processing circuit and the processing circuit in an arbitrary unit, and stores the image signal in the memory means. When writing data, it is possible to compress the image data, and when reading the image data from the memory means, it is possible to expand the image data, and by using this, the data is transferred between the memory means and the signal processing circuit. An imaging apparatus characterized by reducing the amount of data.   2. The imaging apparatus according to claim 1, wherein the compression / decompression processing performed in an arbitrary unit is a reversible compression / decompression processing by DPCM and variable-length encoding processing.   The imaging apparatus according to claim 1 or 2, wherein in the compression processing performed in an arbitrary unit, resolution conversion of a reference image for the processing is performed by paying attention to a frequency resolution necessary for the processing. An image pickup apparatus characterized by performing a compression process after reducing the amount of image data and performing temporary storage in the memory means.   3. The image pickup apparatus according to claim 1, wherein, in the compression processing performed in an arbitrary unit, the amount of image data is reduced by roughening the quantization based on the bit accuracy required for the processing. An image pickup apparatus characterized in that the image data is compressed and temporarily stored in the memory means.

Images