JPH04358476A - Picture data encoder - Google Patents

Abstract

PURPOSE:To attain high picture quality processing and high speed processing by applying optimum encoding over the entire pattern. CONSTITUTION:The picture data encoder in which a picture signal outputted from a solid-state image pickup element 23 is converted into a picture data in a form of a chrominance signal or a luminance signal and a color difference signal and coding processing is applied to the picture data is provided with an extraction means 27 extracting a high frequency component of the picture data and discrimination means 44, 45 discriminating the property of the pattern based on the high frequency component of the picture data outputted from the extraction means 27 and also a revision means 46 revising the coding characteristic of the picture data based on the result of discrimination of the discrimination means 44, 45.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data encoding apparatus for compressing and encoding still image data, moving image data, etc. in, for example, video telephones and electronic still cameras.

0002

2. Description of the Related Art As is well known, an image data encoding system for an electronic still camera has conventionally been used as shown in FIG.
It is configured as shown in a). First, the photographed optical image of the subject is photoelectrically converted by a CCD (charge coupled device) 11 as a solid-state image sensor, and then
The image data is converted into a luminance signal and a color difference signal as image data by the imaging circuit 12 and the signal processing circuit 13, and is recorded in the frame memory 14. Thereafter, the image data recorded in the frame memory 14 is sequentially read out, subjected to data compression processing by a compression encoding circuit 15, and then recorded in a memory card 16 containing a built-in semiconductor memory.

Furthermore, in the case of a videophone, the image data is compressed by a compression encoding circuit 15, as shown in FIG. 15(b), and then transmitted via a telephone line, etc. (not shown). .

By the way, when encoding image data,
The amount of code tends to increase for detailed patterns and decrease for smooth images, so the amount of transmitted codes may change for each image, and the memory capacity for recording image data may increase. The inconvenience of change arises. Therefore,
In order to make the amount of code for each image constant, it has conventionally been considered to prepare a buffer memory at the output stage and control the encoding characteristics based on the usage amount of this buffer memory.

That is, as shown in FIG. 16(a), the image data read out from the frame memory 14 is passed through a quantization circuit 17 and a DPCM (differential encoding) circuit 18, and then sent to a variable length encoding circuit. 19, the signal is subjected to variable length encoding processing such as Huffman coding, and is output via the buffer memory 20. At this time, the output of the buffer memory 20 is fed back to the quantization circuit 17, and the quantization characteristics are changed according to the residual data of the buffer memory 20, so that the output data rate is controlled to be constant.

FIG. 16(b) shows another example using a buffer memory. That is, the image data read from the frame memory 14 passes through the DCT (discrete cosine transform) circuit 21 and the quantization circuit 17, and then
A variable length encoding circuit 19 performs variable length encoding processing such as a Huffman code, and outputs the signal via a buffer memory 20 . At this time, the output of the buffer memory 20 is fed back to the quantization circuit 17, and the quantization characteristics of the DCT output data are changed according to the usage amount of the buffer memory 20, and the output data rate is controlled to be constant.

However, in the conventional image data encoding system as described above, the output data rate is adjusted while encoding the image data, so it is impossible to perform optimal adjustment over the entire screen. A problem has arisen. For example, if the top half of the screen is flat and the bottom half has a detailed pattern, because of sequential processing, excess data will be used for the flat area and not enough data will be allocated to the bottom half, resulting in degraded image quality. It turns out.

[0008] Conventionally, therefore, a method has been considered in which the entire screen is encoded once, the amount of code is checked, and the encoding process is performed again to bring the output data rate close to a constant level. It is necessary to perform the encoding process repeatedly, which makes high-speed and real-time operation impossible.

[0009]

[Problems to be Solved by the Invention] As described above, in the conventional image data encoding device, the output data rate can be kept constant by sequentially monitoring the usage amount of the buffer memory provided in the output stage and changing the encoding characteristics. Therefore, there is a problem in that optimization over the entire screen cannot be desired. Furthermore, the method of repeating the encoding process to keep the output data rate constant has the disadvantage that high-speed processing, particularly real-time processing, cannot be performed.

[0010] Therefore, the present invention was made in consideration of the above circumstances, and by determining the encoding characteristics by looking at the entire screen based on the image data before being encoded,
It is an object of the present invention to provide an extremely good image data encoding device that can achieve high image quality by performing optimal encoding over the entire screen, and can also perform high-speed processing.

[0011]

[Means for Solving the Problems] An image data encoding device according to the present invention includes a solid-state image sensor that outputs an image signal according to the amount of incident light, and a color signal or a luminance signal that converts the image signal output from the solid-state image sensor into a color signal or a luminance signal. , a conversion means for converting into image data in the form of a color difference signal, and an encoding means for performing encoding processing on the image data output from the conversion means. and an extracting means for extracting a high-frequency component of the image data output from the converting means, a determining means for determining the nature of the picture on the screen based on the high-frequency component of the image data output from the extracting means, and and changing means for changing the encoding characteristics of the encoding means based on the determination result of the encoding means.

0012

[Operation] According to the above configuration, the encoding characteristics for the entire screen can be determined based on the high-frequency components of the image data before being encoded. Optimal encoding can be performed to achieve high image quality, and high-speed processing is also possible.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings. In FIG. 1, 22 is a lens system, and an optical image incident through this lens system 22 is photoelectrically converted by a CCD 23, and then converted into digital data by an A/D (analog/digital) conversion circuit 24. Ru. The output data of this A/D conversion circuit 24 is WB(
A brightness signal YLH with correct white balance, a brightness signal YH with incorrect white balance but a wide band, and a color difference signal CRH are supplied to the matrix circuit 26 via the gamma (white balance) correction circuit 25. , CBH are generated.

Of these, the luminance signal YH is passed through an HPF (high pass filter) circuit 27 to remove the high frequency component YH.
HH is taken out, and the luminance signal YLH is passed through an LPF (low pass filter) circuit 28 to convert it into a low frequency component YL.
is taken out. And these luminance signals YHH, YL
are added in the adder circuit 29 to produce a luminance signal Y
is generated and supplied to the frame memory 30. Further, the color difference signals CRH and CBH are supplied to LPF circuits 31 and 3, respectively.
2, color difference signals R-Y and B-Y are generated and supplied to the frame memory 30.

As shown in FIG. 2, the luminance signal Y is read out from the frame memory 30 in blocks of 64 pixels in total, 8 pixels in the horizontal direction and 8 pixels in the vertical direction. The read luminance signal Y is subjected to two-dimensional discrete cosine transform processing in block units by the DCT circuit 33, and the transform coefficients are sent to the zigzag scan circuit 3.
4, it is converted into one-dimensional data. Note that FIG. 3 shows the order in which the coefficients are compared. 0 indicates a DC (direct current) component, and 1 to 63 indicate an AC (alternating current) component.

The coefficients converted into one-dimensional data are quantized in a quantization circuit 35, and the DC component is converted into a DPCM code by taking the difference from the previous block in a DPCM circuit 36, and then sent to a Huffman encoding circuit. Huffman table 3 at 37
Huffman encoded based on 8 and multiplexer 39
is output to. Further, the AC component is Huffman encoded by a Huffman encoding circuit 40 based on a Huffman table 41, and then an EOB code indicating the end of the block is added by an EOB adding circuit 42 and output to the multiplexer 39. The multiplexer 39 selectively outputs the DC component output from the Huffman encoding circuit 37 and the AC component output from the EOB addition circuit 42, and compresses the image data.

In this case, the amount of code can be controlled by changing the quantization characteristics of the quantization circuit 35. For example, if you quantize with a large value, the number of types of numbers after quantization will decrease,
Furthermore, the number of zero coefficients increases and the amount of code decreases. For this reason, a quantization table 43 is created in which quantization values are determined in advance for each coefficient and tabulated, and the amount of code is controlled by multiplying the value of this quantization table 43 by a constant.

That is, in the process of recording the previously created luminance signal Y and color difference signals R-Y, B-Y in the frame memory 30, the absolute value of the high-frequency luminance signal YHH generated by the HPF circuit 27 is After the block addition circuit 44 adds the data block by block, the frame addition circuit 45 calculates the absolute value sum TB for one screen. The value of this absolute value sum TB is
This value serves as an index indicating the nature of the image, that is, whether it is flat or has a fine pattern, and the quantization table 43 is created based on this value.

Furthermore, based on the outputs of both the block addition circuit 44 and the frame addition circuit 45, the bit allocation circuit 46
The bit allocation is set in , and the bit number is calculated in the bit number calculation circuit 47 based on this bit allocation and the outputs of the Huffman encoding circuits 37 and 40 .
2.

The relationship between the absolute value sum TB and the coefficient α that determines how many times the quantization table 43 is to be multiplied is set in advance from a plurality of images, as shown in FIG. In Figure 4, N1~
N3 indicates the code amount for one screen, and indicates that, for example, when the code amount is N2 and the absolute value sum TB is n1, the coefficient α may be set to α1.

[0021] Here, even within one screen, flat parts and parts with fine patterns coexist. Then, by assigning fewer codes to flat areas and increasing codes to areas with detailed patterns, the overall image quality of the screen improves. Therefore,
The bit allocation circuit 46 allocates the amount of code for each block. That is, if the absolute value sum TB of the luminance signal YHH of a certain block is TBBn and the total code amount is N2, the code amount Bbn allocated to that block is Bbn=N2×(TBBn/TB).

This block is arranged in the order shown in FIG.
When encoding (DC component), 1, 2, . . . , if the code amount Bbn allocated to the block is exceeded, the encoding is terminated, and an EOB code indicating the end of the block is added and output. In reality, it is better to always send the EOB code and DC component, so the code amount Bbn is Bbn=(
N2 - EOB code amount - total DC code amount) x (TBBn/T
B)+Block DC code amount.

Furthermore, in the process of sequentially encoding each block, if the allocated code amount is not all used up, the remaining amount is carried over to the next block and added to the allocated code amount of the next block. In addition, in order to control the total code amount more accurately, the actually output code amount is cumulatively added,
It is also possible to modify the code amount for each block.

FIG. 5 shows in detail the portion in FIG. 1 where the luminance signal Y and color difference signals RY and BY are generated from the optical image. That is, to describe the same parts as those in FIG. This CCD 23 has an image forming surface, for example, as shown in FIG.
A color filter is formed as shown in SSG.
A photoelectric conversion operation is performed based on the output of the CCD drive circuit 50 which is controlled by the output of the circuit 49. And C.C.D.
The output of 23 is supplied to the A/D conversion circuit 24 via the noise reduction amplifier circuit 51 and converted into digital data in the order of G, R, G, B, G, R, . . . The circuit 25 performs WB adjustment and γ correction for each of R, G, and B.

Here, the output of this WB adjustment γ correction circuit 25 is
(Horizontal scanning period) Delay circuit 52, 1 sample delay circuit 5
3, 54 and the matrix circuit 55, as shown in FIG. 7, YH = 0.25 (G1 + G2 + R1 + B1) YLH =
0.3(R1-G1)+0.11(B1-G2)+G2 CRH=0.7(R1-G1)-0.11(B1-G2
) CBH=-0.30(R1-G1)+0.89(B1
-G2) The above-mentioned luminance signals YH, YLH
and color difference signals CRH and CBH are generated.

As mentioned above, the luminance signal YH
The high frequency component YHH is extracted by the HPF circuit 27, the low frequency component YL of the luminance signal YLH is extracted by the LPF circuit 28, and the luminance signal Y is generated by adding both components YHH and YL in the adding circuit 29. Ru. Further, the color difference signals CRH and CBH are generated by extracting low frequency components by LPF circuits 31 and 32, respectively.
-Y.

Here, FIG. 8 shows the band of the luminance signal Y. In other words, the horizontal band of the luminance signal YL is fHL
, the vertical band is fVL, the horizontal band of the luminance signal YHH is fHL to fH, and the vertical band is fVL to fV. For example, fH = 7MHz, fHL = 3.5MHz, f
It is assumed that V = 480 TV lines and fVL = 240 TV lines.

Next, FIG. 9 shows the block addition circuit 44.
, details of the frame addition circuit 45 and quantization table 43 are shown. First, the cumulative addition circuit 56 in the block addition circuit 44 calculates the intra-block sum TBBn of the absolute values of the luminance signal YHH for each pixel. This block inner sum T
BBn is expressed as TBBn=Σ(k=0 to 63) |Pk|. However, Pk is the value of the luminance signal YHH of each pixel.

Then, the calculated inner block sum TBBn
is stored in the memory 57, and the cumulative addition circuit 58 in the frame addition circuit 45 adds the intra-block sum TBBn of all blocks to obtain the absolute value sum TB of the luminance signals YHH of one screen. This absolute value sum TB is
It is expressed as TB=Σ(n=0 to M)TBBn. However, M is the total number of blocks on one screen. The calculated absolute value sum TB is then stored in the memory 59.

On the other hand, the ROM (read only memory) 60 in the quantization table 43 is accessed using the absolute value sum TB calculated by the cumulative addition circuit 58 to output the coefficient α. This coefficient α is determined by the first quantization table 61
The actual quantization coefficient is calculated by multiplying the obtained quantization coefficient by the multiplication circuit 62, and the second quantization table 63 is created based on this.

Furthermore, block bit allocation for controlling the code amount for each block is performed by an allocation ratio circuit 64. This allocated bit number Bbn is expressed as Bbn=N×(TBBn/TB). However, N is the total code amount (total number of bits). Here, the absolute value sum TB is the sum of the absolute values of the luminance signal YHH for one screen, and the intra-block sum TBBn is the sum of the absolute values of the luminance signal YHH of block n. On the other hand, if the number of allocated bits is left over in the previous block, it is carried over to the next block, so if this number of leftover bits is set as Brn-1, then the bit allocation Bbn becomes: Bbn=N×(TBBn/TB)+Brn-1.

Next, control of the number of output bits will be explained using FIG. 10. For example, as shown in FIG. 11, when one block of Huffman codes is sent in bit order, the amount of codes (number of bits) is added by an adder circuit 65 shown in FIG. This added value is compared with the number of block allocation bits Bbn by the comparator circuit 66, and when it exceeds the block allocation bit number Bbn - the number of EOB code bits, the Huffman code at the time of exceeding is canceled by the aborting circuit 67. A code is sent, and an EOB code is added to this by an EOB adding circuit 68.

If the number of bits actually used in a block is less than the block allocated bit number Bbn, the remaining bits are carried over to the next block as carryover bits Brn. The above operations are performed until all blocks are completed. At this time, an error of several bits occurs in the last block, but the total code amount is approximately N bits.

The number of bits allocated to each block is determined by the luminance signal YHH, and the color difference signals RY and BY are also allocated in the same manner. When the color difference signals R-Y, B-Y have fewer samples than the luminance signal Y, for example, when Y:RY:BY=4:2:2, two blocks of the luminance signal Y are the color difference signal. R-Y
, B-Y.

Therefore, for example, the allocation of blocks in the color difference signal RY is as follows: Bbn=N'×[(TBBn+TBBn+
1)/2]×(1/TB). However, N'
is the total number of bits of the color difference signal RY, TBBn, TB
Bn+1 is the broadband component Y in the block of the luminance signal Y
It is the sum of the absolute values of H, and two blocks n and n+1 constitute one block of the color difference signal RY. Note that the color difference signal B-
Y is calculated in the same way. Of course, it is also possible to create high-frequency components of the color difference signals RY and BY and use the sum of their absolute values to determine the number of bits to be allocated to the color difference.

Further, in the above embodiment, the sum of absolute values of the luminance signal YHH was used for the quantization characteristic and bit allocation, but as shown in FIG. It is also possible to calculate the sum of absolute values after doing this. For example, FIG. 13 shows the relationship between |YHH| and the weighted result |YHH′|. This conversion can be predetermined to improve image quality from a plurality of images.

Next, FIG. 14 shows another embodiment of the present invention. That is, the WB adjustment γ correction circuit 25
The adjustment circuit 25a and the γ correction circuit 25b are separated, and the color signals R, G, and B after WB adjustment and before γ correction are supplied to the matrix circuit 70 to generate the luminance signal YH. This is a different part from the embodiment shown in FIG. The reason for this is that when a luminance signal and a color difference signal are created from the color signals R, G, and B after γ correction, part of the luminance signal leaks into the color difference signal and if the band is limited, the resolution deteriorates. That is, this is because the constant brightness principle is not satisfied. Therefore, the high-frequency component YH of the luminance signal is created using the color signals R, G, and B before γ correction, and this component YH is used to determine the quantization characteristic and perform block bit allocation. It should be noted that the present invention is not limited to the above-described embodiments, and can be implemented with various modifications without departing from the gist thereof.

[0038]

[Effects of the Invention] As detailed above, according to the present invention,
Since the encoding characteristics are determined by looking at the entire screen based on the image data before encoding processing, it is possible to perform optimal encoding over the entire screen and achieve high image quality, as well as high-speed processing. It is possible to provide an extremely good image data encoding device that is also capable of processing.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of an image data encoding device according to the present invention.

FIG. 2 is a diagram shown to explain blocks of the same embodiment.

FIG. 3 is a diagram showing the order in which transform coefficients of discrete cosine transform are arranged in the same embodiment.

FIG. 4 is a diagram showing the relationship between the intra-frame absolute value sum TB of luminance signals and the coefficient α.

FIG. 5 is a block diagram showing in detail a portion that generates luminance and color difference signals from an optical image.

FIG. 6 is a diagram shown to explain a CCD color filter.

FIG. 7 is a diagram shown to explain calculation processing of luminance and color difference signals.

FIG. 8 is a diagram shown to explain the band of a luminance signal.

FIG. 9 is a block configuration diagram showing details of a part that controls quantization characteristics and bit allocation.

FIG. 10 is a diagram shown to explain details of controlling the number of output bits.

FIG. 11 is a diagram shown to explain the operation of controlling the number of output bits.

FIG. 12 is a block configuration diagram showing another example of a part that controls quantization characteristics and bit allocation.

FIG. 13 is a diagram showing characteristics in another example.

FIG. 14 is a block diagram showing another embodiment of the invention.

FIG. 15 is a block diagram showing an image data encoding system.

FIG. 16 is a block configuration diagram shown to explain conventional problems.

[Explanation of symbols]

11...CCD, 12...imaging circuit, 13...signal processing circuit,
14...Frame memory, 15...Compression encoding circuit, 16...
Memory card, 17... Quantization circuit, 18... DPCM circuit, 19... Variable length encoding circuit, 20... Buffer memory, 2
1...DCT circuit, 22...lens system, 23...CCD, 24
...A/D conversion circuit, 25...WB adjustment γ correction circuit, 26...
Matrix circuit, 27... HPF circuit, 28... LPF circuit, 29... Addition circuit, 30... Frame memory, 31, 32
...LPF circuit, 33...DCT circuit, 34...Zigzag scan circuit, 35...Quantization circuit, 36...DPCM circuit, 3
7...Huffman encoding circuit, 38...Huffman table, 3
9... Multiplexer, 40... Huffman encoding circuit, 41
... Huffman table, 42 ... EOB addition circuit, 43 ... quantization table, 44 ... block addition circuit, 45 ... frame addition circuit, 46 ... bit allocation circuit, 47 ... bit number calculation circuit, 48 ... optical LPF, 49 ... SSG circuit , 50...
CCD drive circuit, 51...low noise amplifier circuit, 52...1H
Delay circuit, 53, 54... 1 sample delay circuit, 55... Matrix circuit, 56... Cumulative addition circuit, 57... Memory, 5
8... Cumulative addition circuit, 59... Memory, 60... ROM, 61
...first quantization table, 62...multiplication circuit, 63...second
quantization table, 64...allocation ratio circuit, 65...addition circuit, 66...comparison circuit, 67...truncation circuit, 68...EOB
Additional circuit, 69... Weighting circuit, 70... Matrix circuit.

Claims (1)

[Claims] 1. A solid-state imaging device that outputs an image signal according to the amount of incident light; a conversion means that converts the image signal output from the solid-state imaging device into image data in the form of a color signal or a luminance or color difference signal; An image data encoding device comprising: encoding means for performing encoding processing on image data output from the converting means; A determining means for determining the nature of a picture on the screen based on a high-frequency component of image data output from the means, and a changing means for changing the encoding characteristic of the encoding means based on the determination result of the determining means. An image data encoding device characterized by: