JP2021090097A - Encoder, imaging apparatus, control method, and program - Google Patents

Abstract

To achieve both encoding considering the feature of an image over a vertical direction and a reduction in circuit scale.SOLUTION: An encoder 150 has: a pixel line buffer 1505 that holds pixel data; a wavelet transformation unit 1500 that executes wavelet transformation on the pixel data to generate a subband coefficient; a quantization unit 1503 that quantizes the subband coefficient in accordance with a quantization parameter; and a quantization control unit 1502 that determines the quantization parameter. The quantization control unit 1502 determines the quantization parameter based on the pixel data held by the pixel line buffer 1505 and corresponding to the subband coefficient to be quantized and pixel data corresponding to a subband coefficient located in a vertical direction of the above-mentioned subband coefficient.SELECTED DRAWING: Figure 15

Description

The present invention relates to a coding device, an imaging device, a control method, and a program for performing quantization and coding on pixel data.

A conventional image pickup device removes noise and corrects optical distortion for a signal indicating brightness and color difference converted by a debayer process (demosaic process) from a raw image (RAW image) captured by an image sensor. , Perform development processing including image optimization. Then, the above imaging device compresses and encodes the developed luminance signal and the color difference signal and records them on the recording medium.

In other conventional image pickup devices, undeveloped data (RAW image) captured by the image pickup device is stored in a recording medium. Since the RAW image is recorded on the recording medium while maintaining the number of color gradations without damaging the color information acquired by the image sensor, editing with a high degree of freedom is possible. However, since the amount of data in the RAW image is larger than the amount of data in the developed image, there is a demerit that the capacity of the recording medium is squeezed. Therefore, it is preferable to perform compression coding on the RAW image to reduce the amount of data.

In the technique of encoding RAW image data (RAW data), two-dimensional wavelet transform is performed on a plurality of color channels generated from RAW data to generate a plurality of subbands, and each subband is quantized. Encode (for example, Patent Documents 1 and 2).

Japanese Unexamined Patent Publication No. 2019-4428 Japanese Unexamined Patent Publication No. 2019-29002

In the technique of Patent Document 1, the quantization parameter is generated based on the subband coefficient of each pixel line existing in the horizontal direction. Therefore, in the generation of quantization parameters, and thus in the coding, horizontal features are considered but vertical features are not.

In the technique of Patent Document 2, the quantization parameter is generated in consideration of the subband coefficient over the vertical direction. However, in the above configuration, quantization and coding cannot be performed until the subband coefficients used to identify the vertical features are obtained. That is, since it is necessary to provide a line buffer for holding the subband coefficient until the quantization parameter is generated, the circuit scale of the coding apparatus becomes bloated.

In view of the above circumstances, it is an object of the present invention to provide a coding device, an imaging device, a control method, and a program capable of achieving both coding in consideration of image features in the vertical direction and suppression of circuit scale. To do.

In order to achieve the above object, the coding apparatus of the present invention uses a buffer means for temporarily holding input pixel data and performing wavelet conversion on the pixel data to generate a plurality of subband coefficients. The conversion means is provided, the quantization means for quantizing the subband coefficient according to the quantization parameter, and the quantization control means for determining the quantization parameter, and the quantization control means is used as the buffer means. The quantization parameter is based on the retained pixel data corresponding to the subband coefficient quantized by the quantization means and the pixel data corresponding to the subband coefficient located in the direction perpendicular to the subband coefficient. It is characterized by determining.

According to the present invention, it is possible to achieve both efficient coding in consideration of the characteristics of the image in the vertical direction and suppression of the circuit scale.

It is a block diagram which shows the structure of a coding apparatus. It is a figure which shows the image data (RAW data) of a Bayer array. It is explanatory drawing of the subband division by a wavelet transform. It is explanatory drawing of the generation method of the feature information using a subband coefficient. It is explanatory drawing of the detection method of a vertical edge using a subband coefficient. It is explanatory drawing of the detection method of a horizontal edge using a subband coefficient. It is explanatory drawing of the basic concept of a feature classification using lightness information and complexity information. It is explanatory drawing of the method of changing a feature classification based on edge information. It is explanatory drawing of the timing which DWT coefficient is output by DWT conversion. It is explanatory drawing of the execution method of the DWT conversion at the edge (left edge) of a screen. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is explanatory drawing of the state of the line buffer in the vertical DWT transform. It is a block diagram which shows the detailed structure of a coding apparatus. It is a timing chart which shows the time series of processing in a coding apparatus. It is explanatory drawing of the detection method of the horizontal edge in 1st Embodiment of this invention. It is a block diagram which shows the structure of the coding apparatus which concerns on 1st Embodiment of this invention. It is a timing chart which shows the time series of the processing in the coding apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the correspondence between the subband coefficient and pixel data in 1st Embodiment of this invention. It is a figure which shows the correspondence between the subband coefficient and pixel data in 1st Embodiment of this invention. It is explanatory drawing of the detection method of a dark part using a subband coefficient. It is explanatory drawing of the dark part detection method using the pixel average value of 2 × 2 pixels in the 1st Embodiment of this invention. It is explanatory drawing of the method of changing a feature classification based on the edge information in 1st Embodiment of this invention. It is explanatory drawing of the detection method of the horizontal edge in 2nd Embodiment of this invention. It is a block diagram which shows the structure of the coding apparatus which concerns on 2nd Embodiment of this invention. It is a timing chart which shows the time series of processing in the coding apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows the correspondence between the subband coefficient and the pixel data in the 2nd Embodiment of this invention. It is a figure which shows the correspondence between the subband coefficient and the pixel data in the 2nd Embodiment of this invention.

Prior to the description of the embodiment of the present invention, the configuration of the coding device 100 and the processing executed by the coding device 100, which are the premise of the embodiment, will be described. In the following description, the coding apparatus 100 performs coding based on the quantization parameters generated in consideration of the characteristics of the subband coefficients in the horizontal direction and the vertical direction.

FIG. 1 is a block diagram showing a configuration of a coding device 100. The coding device 100 includes an imaging unit 101, a plane conversion unit 102, a wavelet conversion unit 103, a feature information generation unit 104, a quantization control unit 105, a quantization unit 106, an entropy coding unit 107, and a recording medium 108.

The imaging unit 101 includes a lens optical system that includes an optical lens, an aperture, a focus control unit, and a lens driving unit and is capable of optical zoom, and an imaging element that converts optical information incident through the lens optical system into an electric signal. .. Therefore, the coding device 100 of this example functions as an imaging device. The image pickup device is, for example, a CCD image sensor or a CMOS sensor, and has a Bayer array color filter having 4 pixels of 2 pixels in the horizontal direction and 2 pixels in the vertical direction as a repeating unit. The image sensor outputs RAW data obtained by converting an electric signal (analog signal) into a digital signal to the plane conversion unit 102. As a non-limiting example herein, RAW data is composed of four color elements of the Bayer sequence shown in FIG. 2, R (red), G1 (green), G2 (green), and B (blue). ..

The plane conversion unit 102 reads the RAW data input from the imaging unit 101 and converts it into four independent plane data. The four plane data obtained by the conversion as described above are output from the plane conversion unit 102 to the wavelet conversion unit 103. The conversion from RAW data composed of four color elements (R, G1, G2, B) to plain data is executed according to equation (1) including the following four equations. According to the equation (1), the RAW data is approximately converted into the luminance plane (Y) and the other planes (C0, C1, C2).

Y = (R + G1 + G2 + B) / 4
C0 = RB
C1 = (G1 + G2) / 2- (R + B) / 2
C2 = G1-G2 …… Equation (1)

The conversion formula between the RAW data and the plane data is not limited to the above formula (1). For example, the RAW data may be separated into R, G1, G2, and B to be plain data.

The wavelet transform unit 103 reads out the plane data input from the plane transform unit 102, performs wavelet transform, and acquires a conversion coefficient (sub-band coefficient). The transform coefficient is a value indicating the correlation between the target data (that is, plain data) and the wavelet. The acquired conversion coefficient is output to the feature information generation unit 104 and the quantization unit 106.

With reference to FIG. 3, the subband division by the wavelet transform described above will be exemplified. FIG. 3 shows a subband corresponding to decomposition level 1 obtained by performing one vertical wavelet transform and one horizontal wavelet transform on plain data. In FIG. 3, the subband LL shows a low frequency component, and the other subbands LH, HL, and HH show a high frequency component.

The feature information generation unit 104 uses the subband coefficients of the Y plane input from the wavelet transform unit 103 to generate three feature information including brightness, complexity, and the presence or absence of edges. Each of the generated feature information is output from the feature information generation unit 104 to the quantization control unit 105. The method for generating feature information will be described later.

The quantization control unit 105 generates a quantization parameter corresponding to the feature information input from the feature information generation unit 104 and outputs the quantization parameter to the quantization unit 106. The relationship between the feature information and the quantization parameter will be described later.

The quantization unit 106 executes the quantization of the conversion coefficient (sub-band coefficient) with respect to the sub-band data input from the wavelet conversion unit 103 by using the quantization parameter input from the quantization control unit 105. The quantized conversion coefficient is output from the quantization unit 106 to the entropy coding unit 107.

The entropy coding unit 107 encodes the conversion coefficient (subband coefficient) quantized in the quantization unit 106 to acquire the coded data. The above compression coding is, for example, entropy coding such as Golomb coding. The coded data is output from the entropy coding unit 107 to the recording medium 108.

The recording medium 108 is, for example, a recording medium composed of a non-volatile memory, and stores coded data input from the entropy coding unit 107.

In the functional blocks from the plane conversion unit 102 to the entropy coding unit 107, one or more processors such as a CPU of the coding device 100 expand the program stored in the non-volatile memory into the volatile memory and execute the program. It is realized by.

Next, the relationship between the feature information and the quantization parameter will be described below. As described above, as the feature information, feature information indicating brightness, feature information indicating complexity, and feature information indicating the presence or absence of edges are exemplified.

The brightness indicates the brightness level of the RAW data. In the quantization process, if coarse quantization (quantization with a small number of quantization bits) is performed on a region with a low luminance level (dark region), the coefficient after quantization tends to be zero. That is, when coarse quantization is applied to the dark region, fine information is likely to be lost. Further, for RAW data, brightness level adjustment such as gamma correction and tone curve correction is executed in processing (post processing) after data recording. In the above luminance level adjustment, in general, the luminance level in the dark region is increased and the luminance level in the bright region is decreased. Therefore, when both the dark region and the bright region are quantized using the same quantization parameter, the degree of amplification of the quantization distortion is large in the dark region where the brightness level is greatly increased, so that the image quality deterioration is conspicuous. Cheap. On the other hand, in the bright region where the brightness level is reduced, the degree of amplification of the quantization distortion is small, so that the deterioration of image quality is not noticeable. Based on the above tendency, it is preferable to finely quantize so that the quantization distortion becomes small in the dark region where the deterioration of image quality is conspicuous, and coarsely quantize the code amount so as not to increase in the bright region where the deterioration of image quality is not conspicuous.

Complexity indicates the height of the spatial frequency of the RAW data. The presence or absence of an edge indicates the presence or absence of an edge, which is a portion where the brightness level changes rapidly. In RAW data, the lower the spatial frequency, the flatter the region, and the higher the spatial frequency, the more complex the region. Generally, in image compression, visual deterioration is easily noticeable when information in a flat region is reduced, while visual deterioration is not noticeable even when information in a complex region is reduced. However, even in a complicated region (a region having a high spatial frequency), in a region including an edge, which is a portion where the brightness level changes rapidly, visual deterioration is likely to be noticeable when information is reduced. Based on the above tendency, it is preferable to finely quantize in a flat region or an edge-containing region where visual deterioration is easily noticeable, and coarsely quantize in a complicated region where visual deterioration is not noticeable.

Next, each feature information generation method will be described below with reference to FIGS. 4 to 6. The feature information generation unit 104 uses the subband coefficient of the Y plane as a feature amount to generate feature information indicating brightness, complexity, and the presence / absence of edges in predetermined block units.

The generation of characteristic information in the horizontal direction will be described with reference to FIG. In this example, the feature information is generated for each block 401 to 404, which is a generation unit of the feature information, by using four conversion coefficients (horizontal four coefficients) that are continuous in the horizontal direction in the subband of the decomposition level 1. Subband formation at decomposition level 1 In FIG. 400, blocks 401 to 404 point to the same spatial coordinates.

As mentioned above, the brightness feature information indicates the luminance level of the RAW data. Therefore, the brightness P1 can be generated by using the block 401 having a 1LL subband coefficient which is a DC component of the image. The brightness P1 is represented by the following equation (2) using the coefficients A0, A1, A2, and A3 in the block 401.

P1 = (A0 + A1 + A2 + A3 + 2) >> 2 …… Equation (2)

Further, the complexity feature information (complexity P2) is generated based on the block 402 of the 1HL subband coefficient, which is the AC component of the image, the block 403 of the 1LH subband coefficient, and the block 404 of the 1HH subband coefficient. Can be done. The complexity P2 is the maximum value of the intermediate evaluation value PHL generated for the block 402, the intermediate evaluation value PLH generated for the block 403, and the intermediate evaluation value PHH generated for the block 404, and is expressed by the following equation. It is represented by (3).

P2 = MAX (PHL, PLH, PHH) …… Equation (3)

The intermediate evaluation value PHL in the formula (3) is represented by the following formula (4) using the coefficients B0, B1, B2, and B3 in the block 402. Similarly, the intermediate evaluation value PLH is represented by the following equation (5) using the coefficients C0, C1, C2, C3 in the block 403, and the intermediate evaluation value PHH is the coefficients D0, D1, D2 in the block 404. , D3 is expressed by the following equation (6). In equations (4) to (6), ABS indicates an absolute value conversion process.

PHL = {ABS (B0) + ABS (B1) + ABS (B2) + ABS (B3) +2} >> 2 …… Equation (4)
PLH = {ABS (C0) + ABS (C1) + ABS (C2) + ABS (C3) +2} >> 2 …… Equation (5)
PHH = {ABS (D0) + ABS (D1) + ABS (D2) + ABS (D3) +2} >> 2 …… Equation (6)

A process of detecting whether or not an edge in the vertical direction (vertical edge) is included in the above-mentioned four horizontal coefficients will be described with reference to FIG. FIG. 5A illustrates the frequency components (horizontal 4 coefficients (n to n + 3)) used for detecting the vertical edge in this example. By executing the detection process using the coefficient of the HL subband with respect to the frequency component in the horizontal direction, the vertical edge, which is the high frequency component in the horizontal direction, can be detected. In the detection process of this example, when each coefficient exceeds a predetermined threshold value, it is determined to be "large amplitude value", while when it is equal to or less than a predetermined threshold value, it is determined to be "small amplitude value". Then, according to the array of the above determination values (“large amplitude value” or “small amplitude value”), it is determined whether or not the vertical edge is included in the horizontal 4 coefficients.

FIG. 5B illustrates an array of horizontal 4 coefficients that are determined to include vertical edges. The horizontal coordinate positions (n to n + 3) on the horizontal axis in FIG. 5 (b) correspond to the horizontal 4 coefficients (n to n + 3) in FIG. 5 (a). The vertical axis in FIG. 5B corresponds to the coefficient value at each horizontal coordinate position. As shown in the figure, the judgment values at the horizontal coordinate position are n = small amplitude value, n + 1 = small amplitude value, n + 2 = large amplitude value, n + 3 = small amplitude value, and the horizontal coordinate position changes from n to n + 3. When doing so, the amplitude value (coordinate value) changes across the threshold. The fact that the amplitude value changes across the threshold value as described above indicates that the horizontal continuity of the image is broken at the portion corresponding to the horizontal 4 coefficient, and therefore it is determined that the vertical edge exists. ..

With reference to FIG. 6, a process for detecting whether or not a horizontal edge (horizontal edge) is included in three vertically continuous conversion coefficients (vertical three coefficients) will be described. FIG. 6A illustrates the frequency components (vertical 3 coefficients (m-1 to m + 1)) used for detecting the horizontal edge in this example. By executing the detection process using the coefficient of the LH subband with respect to the frequency component in the vertical direction, the horizontal edge, which is the high frequency component in the vertical direction, can be detected. In the detection process of this example, when each coefficient exceeds a predetermined threshold value, it is determined to be "large amplitude value", while when it is equal to or less than a predetermined threshold value, it is determined to be "small amplitude value". Then, according to the array of the above determination values (same as above), it is determined whether or not the horizontal edge is included in the vertical three coefficients.

FIG. 6B illustrates an array of vertical 3 coefficients that are determined to include horizontal edges. The vertical coordinate positions (m-1 to m + 1) on the horizontal axis in FIG. 6B correspond to the vertical 3 coefficients (m-1 to m + 1) in FIG. 6A. The vertical axis in FIG. 6B corresponds to the coefficient value at each vertical coordinate position. As shown in the figure, the judgment values at the vertical coordinate position are m-1 = small amplitude value, m = large amplitude value, and m + 1 = small amplitude value, respectively, when the vertical coordinate position changes from m-1 to m + 1. There is a change in the amplitude value (coordinate value) across the threshold. The fact that the amplitude value changes across the threshold value as described above indicates that the vertical continuity of the image is broken at the portion corresponding to the vertical three coefficients, so that it is determined that the horizontal edge exists. ..

With reference to FIG. 7, the basic concept of feature classification executed by the quantization control unit 105 will be described. In the feature classification table shown in FIG. 7, the threshold value TH0 is a threshold value for classifying the luminance P1 into either low-luminance or medium-luminance, and the threshold TH1 indicates that the luminance P1 is classified into either medium-luminance or high-luminance. It is a threshold to classify. Further, the threshold value TH2 is a threshold value for classifying the complexity P2 indicating the spatial frequency into either low frequency or medium frequency, and the threshold value TH3 is a threshold value for classifying the complexity P into either medium frequency or high frequency. Is. The above four thresholds TH0 to TH3 are parameters that can be arbitrarily set according to conditions such as the bit depth of the image. The characteristics of each block are classified according to the magnitude relationship between the above four thresholds TH0 to TH3, the brightness P1, and the complexity P2.

The correction amounts Q0 to Q8 are correction amounts of quantization parameters to be applied to the feature information (luminance P1, complexity P2) classified based on the threshold values TH0 to TH3. The regions to which the correction amounts Q0 to Q8 should be applied may be referred to as regions Q0 to Q8 using the same reference numerals. The magnitude relationship between the correction amounts Q0 to Q8 is illustrated by the following equation (7).

Q0 <Q1 ≦ Q3 ≦ Q2 ≦ Q4 ≦ Q6 <Q5 ≦ Q7 <Q8 …… Equation (7)

It is preferable that the quantization parameter is weighted so as to be the smallest in the low-luminance and flat region Q0 and the largest in the high-luminance and complex region Q8 in consideration of visual characteristics. The magnitude relation of the correction amount in the equation (7) is set based on the distance from the region Q0 to each of the other regions Q1 to Q7.

The feature classification considering the existence of the edge will be described with reference to FIG. When it is determined that an edge exists in the above-mentioned determination of the vertical edge and the horizontal edge, the feature classification table shown in FIG. 7 is changed as shown in FIG. For example, the quantization control unit 105 changes the elements classified into the regions Q3 and Q6 (region 801) so as to be classified into the region Q0 (region 802). Similarly, the quantization control unit 105 sets the elements classified into the regions Q4 and Q7 (region 803) into the region Q1 (region 804) and the elements classified into the regions Q5 and Q8 (region 805) into the region Q2 (region Q2). It is changed so that it is classified into 806). According to the above configuration, the quantization parameter of the edge region where visual deterioration is easily noticeable can be kept low.

Next, the timing of the wavelet transform and the line buffer will be described below with reference to FIGS. 9 to 11. In the following example, the wavelet transform unit 103 executes wavelet transform (hereinafter, referred to as DWT transform) using a reversible 5-3 tap filter.

The timing at which the wavelet coefficient (hereinafter referred to as the DWT coefficient) is output will be described with reference to FIG. The DWT coefficient (subband coefficient) of this example is obtained by the DWT transform using the lifting structure. Pixel data is sequentially added as the state of FIG. 9A transitions to the state of FIG. 9B and the state of FIG. 9C.

In FIG. 9A, the input pixel data a to e correspond to pixels continuously arranged in the horizontal direction. The DWT coefficient b _H is a coefficient of the high frequency component of the decomposition level 1 generated according to the following equation (8-1) using the pixel data a, b, and c. The DWT coefficient d _H is a coefficient of the high frequency component of the decomposition level 1 generated according to the following equation (8-2) using the pixel data c, d, and e. The DWT coefficient c _L is a coefficient of the low frequency component of the decomposition level 1 generated according to the following equation (8-3) using the _{DWT coefficient b H} , the DWT coefficient d _{H, and the pixel data c.}

b _H = b− (a + c) / 2 …… Equation (8-1)
d _H = d− (c + e) / 2 …… Equation (8-2)
c _L = c + (b _H + d _H + 2) / 4 …… Equation (8-3)

FIG. 9B shows a state in which pixel data f is newly input in the horizontal direction with respect to the state of FIG. 9A. As can be understood from FIG. 9B, even if one pixel data f is added, the number of pixel data is insufficient to execute the new DWT conversion, so that the new DWT coefficient is not output.

FIG. 9C shows a state in which pixel data g is newly input in the horizontal direction with respect to the state of FIG. 9B. As can be understood from FIG. 9 (c), since the two pixel data f and g are added as compared with FIG. 9 (a), a new DWT transform can be executed. As a result, the DWT coefficient f _H of the high frequency component and the DWT coefficient e _L of the low frequency component are output.

As described above, every time the pixel data for two pixels is input in the horizontal direction, the DWT conversion in the horizontal direction (hereinafter, referred to as horizontal DWT conversion) is executed once. The above relationship is the same in the vertical direction, and one vertical DWT conversion (hereinafter referred to as vertical DWT conversion) is executed every time pixel data for two pixels is input in the vertical direction. ..

Regarding the number of pixel data, as shown in FIG. 3, the size of each subband of decomposition level 1 is half of the input pixel data in each of the horizontal direction and the vertical direction. Therefore, the number of each subband coefficient is 1/4 of the number of pixels of the original image.

A method of executing the DWT conversion at the edge (left edge) of the screen will be described with reference to FIG. In FIG. 10, the three input pixel data a, b, and c correspond to pixels continuously arranged in the horizontal direction from the left edge of the screen. As shown in FIG. 10, when the horizontal DWT conversion is executed at the left edge of the screen, the missing pixel data is obtained by copying the pixel data b and c so as to be symmetrical with respect to the pixel data a closest to the left edge. Extend (pixel data that should be located to the left of the left edge). With the above data expansion, the horizontal DWT transform can be executed even at the left end. The horizontal DWT transform can be similarly executed at the right end. Further, the vertical DWT transform can be similarly performed at the upper end and the lower end.

The vertical DWT transform using the lifting structure will be described with reference to FIGS. 11A to 11G. In the DWT transform of this example, pixel data for one line is sequentially input. Pixel data a to g in FIGS. 11A to 11G show input pixel data for one line. In the time series from FIG. 11A to FIG. 11G, the data stored in the pixel line buffer and the vertical DWT line buffer changes as the pixel data a to g are input. The pixel line buffer buffers pixel data for one line in order to calculate the DWT coefficient of the low frequency component and the DWT coefficient of the high frequency component. The vertical DWT line buffer buffers the DWT coefficient of one line of high frequency components in order to calculate the DWT coefficient of the low frequency components.

As shown in FIG. 11A, when the pixel data a for one line is input, the wavelet transform unit 103 stores the pixel data a in the pixel line buffer Line_Buffer0.

As shown in FIG. 11B, when the pixel data b for the next one line is input, the wavelet transform unit 103 stores the pixel data b in the pixel line buffer Line_Buffer1.

As shown in FIG. 11C, when the pixel data c for the next one line is input, the wavelet transform unit 103 uses the pixel data c and the pixel data a and b of the pixel line buffer Line_Buffer0 and 1 to perform the decomposition level. calculating the DWT coefficients _{b H} the first high frequency component. Further, the wavelet transform unit 103 calculates _{the DWT coefficient a L} of the low frequency component of the decomposition level 1 by using _{the calculated DWT coefficient b H} and the pixel data a stored in the pixel line buffer Line_Buffer0. _{After calculating the DWT coefficient a L} , the wavelet transform unit 103 overwrites the image data a of the pixel line buffer Line_Buffer0 with the image data c. The wavelet transform unit 103 outputs the calculated DWT coefficient a _L to the horizontal DWT transform unit that executes the horizontal DWT transform. The wavelet transform unit 103, the calculated DWT coefficients _{b H,} and outputs to the horizontal DWT transform unit, and stores the vertical DWT line buffer Line_Buffer2.

As shown in FIG. 11D, when the pixel data d for the next one line is input, the wavelet transform unit 103 overwrites the image data b of the pixel line buffer Line_Buffer1 with the image data d.

As shown in FIG. 11E, when the pixel data e for the next one line is input, the wavelet transform unit 103 uses the pixel data e and the pixel data c and d of the pixel line buffer Line_Buffer0 and 1 to perform the decomposition level. _{The DWT coefficient d H} of the high frequency component of 1 is calculated. Furthermore, the wavelet transform unit 103, calculated DWT coefficients _{d H} and DWT coefficients of DWT coefficients _{b H} and a low-frequency component of the decomposition level 1 with the pixel data c and the vertical DWT line buffer Line_Buffer2 pixel line buffers Line_Buffer0 c Calculate _{L. After calculating the DWT coefficient c L} , the wavelet transform unit 103 overwrites the image data c of the pixel line buffer Line_Buffer0 with the image data e. The wavelet transform unit 103 outputs the calculated DWT coefficient c _L to the horizontal DWT transform unit. The wavelet transform unit 103, the calculated DWT coefficients _{d H,} and outputs to the horizontal DWT transform unit, and stores the vertical DWT line buffer Line_Buffer3.

As shown in FIG. 11F, when the pixel data f for the next one line is input, the wavelet transform unit 103 overwrites the image data d of the pixel line buffer Line_Buffer1 with the image data f.

As shown in FIG. 11G, when the pixel data g for the next one line is input, the wavelet transform unit 103 uses the pixel data g and the pixel data e and f of the pixel line buffer Line_Buffer0 and 1 to perform the decomposition level. _{The DWT coefficient f H} of the high frequency component of 1 is calculated. Furthermore, the wavelet transform unit 103, the calculated DWT coefficients _{f H} and DWT coefficients of low frequency components at the decomposition level 1 with the DWT coefficients _{d H} pixel data e and the vertical DWT line buffer Line_Buffer2 pixel line buffers Line_Buffer0 e Calculate _{L. After calculating the DWT coefficient e L} , the wavelet transform unit 103 overwrites the image data e of the pixel line buffer Line_Buffer0 with the image data g. The wavelet transform unit 103 outputs the calculated DWT coefficient e _L to the horizontal DWT transform unit. The wavelet transform unit 103, the calculated DWT coefficients _{f H,} and outputs to the horizontal DWT transform unit, and overwrites the DWT coefficients _{b H} stored in the vertical DWT line buffer Line_Buffer2.

After that, the wavelet transform unit 103 repeatedly executes the same process until the pixel data of the final line is reached. As can be seen from the above, in order to execute the vertical DWT transform of this example, at least two lines of pixel line buffers Line_Buffer0 and 1 and two lines of vertical DWT line buffers Line_Buffer2 and 3 are required.

FIG. 12 is a block diagram illustrating a detailed configuration of the coding apparatus 100 capable of executing the above-mentioned processing after the plane conversion. The coding device 100 includes a wavelet transform unit 1200, a feature information generation unit 1201, a quantization control unit 1202, a quantization unit 1203, and an entropy coding unit 1204. The above elements correspond to the wavelet transform unit 103, the feature information generation unit 104, the quantization control unit 105, the quantization unit 106, and the entropy coding unit 107 of FIG. 1, respectively. It is preferable that the elements (not shown) in FIG. 12 are configured in the same manner as in FIG.

The wavelet transform unit 1200 has a pixel line buffer 1205, a vertical DWT transform unit 1206, a vertical DWT line buffer 1207, a horizontal DWT transform unit 1208, and a horizontal DWT line buffer 1209 used for DWT transform of decomposition level 1. The feature information generation unit 1201 has a horizontal edge determination line buffer 1210 used for determining a horizontal edge.

FIG. 13 is a timing chart showing a time series of DWT transform, feature information generation, and quantization and entropy coding performed in the coding apparatus 100 shown in FIG. In this example, the coding apparatus 100 executes vertical DWT conversion and horizontal DWT conversion up to decomposition level 1 on the pixel data after plane conversion input in raster order.

The horizontal axis of FIG. 13 shows the transition of time. In the periods t0 to t10, pixel data a to k for one line are input, respectively. The input of image data starts from the period t0. Hereinafter, the rectangular bar (hereinafter, simply referred to as “bar”) shown in each line will be described.

Bars a to k shown in the "pixel data" line indicate pixel data a to k for one line input to the pixel line buffer 1205 during the corresponding period. As shown in the figure, since the bar is shown in the "pixel data" line over the entire period, one line of pixel data is input to the pixel line buffer 1205 over the entire period. For example, in the period t0, one line of pixel data a is input to the pixel line buffer 1205. Further, in the period t1, the pixel data b for one line is input to the pixel line buffer 1205.

The bars a to j shown in the "pixel line buffer" line indicate that the input pixel data for one line is temporarily held in the pixel line buffer 1205 during the corresponding period. For example, in the period t1, the pixel data a is held in the pixel line buffer Line_Buffer0. Further, in the period t2, the pixel data a and b are held in the pixel line buffers Line_Buffer0 and 1, respectively.

_{The bars a L to} i _L and b _{H to} j _H shown in the "vertical DWT transform" line ("Lev1-L" line and "Lev1-H" line) are set to the decomposition level 1 by the vertical DWT transform unit 1206 during the corresponding period. Indicates that the vertical DWT transform of is performed. The above DWT coefficient data after vertical DWT conversion is output to the vertical DWT line buffer 1207 or the horizontal DWT conversion unit 1208. For example, in period t1, no vertical DWT transform is performed. Further, in the period t2, the vertical DWT transform of the decomposition level 1 is executed, and the DWT coefficients a _L and b _H are output.

_{For the bars b H} , d _H , f _H , and h _H shown in the "Vertical DWT line buffer" line, the DWT coefficient from the vertical DWT converter 1206 is temporarily held in the vertical DWT line buffer 1207 during the corresponding period. Indicates that you are. For example, in the period t3, DWT coefficients _{b H} for one line is held in a vertical DWT line buffer Line_Buffer2. Further, in the period t5, DWT coefficients _{b H} for one line are held in a vertical DWT line buffer Line_Buffer2, DWT coefficients _{d H} of one line is held in a vertical DWT line buffer Line_Buffer3.

_{Bars a XX to} j _XX (X is L or L or) shown in the "horizontal DWT transform" line ("Lev1-LL" line, "Lev1-HL" line, "Lev1-LH" line, "Lev1-HH" line). H) indicates that the decomposition level 1 horizontal DWT transform is performed during the corresponding period. The above horizontal DWT conversion is executed by the horizontal DWT conversion unit 1208. The DWT coefficient data after the horizontal DWT conversion is output to the horizontal DWT line buffer 1209. For example, in the periods t0 and t1, the horizontal DWT transform is not performed. Further, in the period t2, the horizontal DWT transform of the decomposition level 1 is executed, and the DWT coefficients a _LL , a _HL , b _LH , and b _HH are output to the horizontal DWT line buffer 1209.

_{The bars a XX to} h _XX shown in the "horizontal DWT line buffer" line indicate that the DWT coefficient from the horizontal DWT converter 1208 is temporarily held in the horizontal DWT line buffer 1209 during the corresponding period. For example, during periods t3 to t4, 0.5 lines of DWT coefficients a _LL , a _HL , b _LH , and b _HH are held in the horizontal DWT line buffer 1209. Further, in the period t5 to t6, the DWT coefficients c _LL , c _HL , d _LH , and d _HH for 0.5 lines are held in the horizontal DWT line buffer 1209.

_{The bars b LH} , d _LH , and f _LH shown in the "horizontal edge determination line buffer" line are such that the LH component of the DWT coefficient from the horizontal DWT line buffer 1209 is temporarily held in the horizontal edge determination line buffer 1210 during the corresponding period. Indicates that it has been done. For example, in the period t5 to t6, the DWT coefficient b _{LH for} 0.5 lines is held in the horizontal edge determination line buffer 1210. Further, in the periods t7 to t8, the DWT coefficient d _{LH for} 0.5 lines is held in the horizontal edge determination line buffer 1210.

_{Bars a XX to} h _XX shown in the "Quantization / entropy coding" line quantize and encode the DWT coefficient data after horizontal DWT conversion by the quantization unit 1203 and the entropy coding unit 1204 during the corresponding period. Show that. For example, in period t4, the DWT coefficients a _LL , a _HL , b _LH , b _HH after the horizontal DWT transform are quantized and entropy encoded. Further, in the period t6, the DWT coefficient data c _LL , c _HL , d _LH , and d _HH after the horizontal DWT transform are quantized and entropy encoded.

The coding process executed by the coding apparatus 100 shown in FIG. 12 during the period shown in FIG. 13 will be described below in chronological order.

In the period t0, the pixel data a for one line is input to the pixel line buffer 1205 of the wavelet transform unit 1200. In the period t1, one line of pixel data b is input to the pixel line buffer 1205. During the above periods t0 and t1, the pixel data of the number of lines required to execute the vertical DWT transform is not input to the wavelet transform unit 1200. Therefore, the input pixel data a and b are held in the pixel line buffers Line_Buffer0 and 1, but the vertical DWT transform is not executed.

In the period t2, the vertical DWT transform unit 1206 calculates the DWT coefficients _{b H} using pixel data a, b and the input pixel data c in the pixel line buffer 1205. Calculated DWT coefficients _{b H} is output to the vertical DWT line buffer 1207 and (Line_Buffer2) and horizontal DWT transform unit 1208. Further, the vertical DWT conversion unit 1206 calculates the DWT coefficient a _L using the pixel data a in the pixel line buffer 1205 and the calculated DWT coefficient b _H, and outputs the DWT coefficient a L to the horizontal DWT conversion unit 1208. The horizontal DWT conversion unit 1208 executes horizontal DWT conversion on _{the input DWT coefficients a L} and b _{H , calculates the DWT coefficients a LL} , a _HL , b _LH , and b _HH, and outputs them to the horizontal DWT line buffer 1209. .. The image data a of the pixel line buffer Line_Buffer0 is overwritten with the pixel data c after the calculation of the _{DWT coefficient a L.}

In the period t3, the pixel data d for one line is overwritten by the pixel line buffer 1205 (Line_Buffer1) holding the pixel data b. Similar to the periods t0 to t1, the vertical DWT transform is not executed because the pixel data of the number of lines required to execute the vertical DWT transform is not input to the wavelet transform unit 1200.

In the period t4, the vertical DWT transform unit 1206 calculates the DWT coefficients _{d H} using the pixel data c, d and the input pixel data e in the pixel line buffer 1205. The calculated DWT coefficient d _H is output to the vertical DWT line buffer 1207 (Line_Buffer3) and the horizontal DWT converter 1208. Further, the vertical DWT conversion unit 1206 calculates the DWT coefficient c _L using the pixel data c in the pixel line buffer 1205 and the calculated DWT coefficient d _H, and outputs the DWT coefficient c L to the horizontal DWT conversion unit 1208. The horizontal DWT transform unit 1208 executes horizontal DWT transform on _{the input DWT coefficients c L} and d _H to calculate the _{DWT coefficients c LL} , c _HL , d _LH and d _HH. At the same time, the horizontal DWT line buffer 1209 outputs the held DWT coefficients a _LL , a _HL , b _LH , and b _HH to the feature information generation unit 1201. The feature information generation unit 1201 generates _{brightness using the DWT coefficient a LL} , generates complexity using the DWT coefficients a _HL , b _LH , and b _HH , and generates vertical edge information using the _{DWT coefficient a HL.} Then, it is output to the quantization control unit 1202. The quantization control unit 1202 generates quantization parameters corresponding to the feature information (brightness, complexity, edge information) from the feature information generation unit 1201 and outputs the quantization parameters to the quantization unit 1203. The quantization unit 1203 uses the quantization parameters input from the quantization control unit 1202 to quantize the DWT coefficients a _LL , a _HL , b _LH , and b _HH held in the horizontal DWT line buffer 1209. The quantized DWT coefficients a _LL , a _HL , b _LH , and b _HH are output from the quantized unit 1203 to the entropy encoding unit 1204.

When processing the upper end portion of the image, since the DWT coefficient of the line located above the upper end does not exist, the determination of the horizontal edge described with reference to FIG. 6 is not executed and the quantization parameter is generated. Will be done. Even when processing the lower end portion of the image, since the DWT coefficient of the line located below the lower end does not exist, the determination of the horizontal edge is not executed in the same manner.

As described above, the horizontal DWT transform unit 1208 _{outputs the DWT coefficients c LL} , c _HL , d _LH , and d _HH and stores them in the horizontal DWT line buffer 1209. Since the DWT coefficient f _LH corresponding to the lower line has not been calculated, the horizontal edge determination using the _{DWT coefficients c LL} , c _HL , d _LH , and d _{HH cannot be performed yet. The DWT coefficients a LL} , a _HL , b _LH , and b _{HH that} are used for quantization and entropy coding and are no longer needed _{are obtained by the DWT coefficients c LL} , c _HL , d _LH , and d _HH output by the horizontal DWT transform unit 1208. It will be overwritten. The DWT coefficient b _LH is stored in the horizontal edge determination line buffer 1210 before overwriting. The image data c of the pixel line buffer Line_Buffer0 is overwritten with the pixel data e after the calculation of the _{DWT coefficient c L.}

In the period t5, the pixel data f for one line is overwritten by the pixel line buffer 1205 (Line_Buffer1) holding the pixel data d. Similar to the periods t0 to t1 and t3, the vertical DWT transform is not executed because the pixel data of the number of lines required to execute the vertical DWT transform is not input to the wavelet transform unit 1200.

In the period t6, the vertical DWT transform unit 1206 calculates the DWT coefficients _{f H} with the pixel data e, f and input pixel data g in the pixel line buffer 1205. Calculated DWT coefficients _{f H} is output to the vertical DWT line buffer 1207 and (Line_Buffer2) and horizontal DWT transform unit 1208. Further, the vertical DWT conversion unit 1206 calculates the DWT coefficient e _L using the pixel data e in the pixel line buffer 1205 and the calculated DWT coefficient f _H, and outputs the DWT coefficient e L to the horizontal DWT conversion unit 1208. Horizontal DWT transform unit 1208, DWT coefficients _e L inputted, running horizontally DWT converted to _{f H,} DWT coefficients _{e LL, e HL, f LH} , to calculate the _{f HH.} The DWT coefficient f _LH is output to the feature information generation unit 1201. At the same time, the horizontal DWT line buffer 1209 outputs the held DWT coefficients c _LL , c _HL , d _LH , and d _HH to the feature information generation unit 1201. In addition, the horizontal edge determination line buffer 1210 outputs the held DWT coefficient b _LH to the feature information generation unit 1201. The feature information generation unit 1201 generates _{brightness using the DWT coefficient c LL} , generates complexity using the DWT coefficients c _HL , d _LH , and d _HH , and generates vertical edge information using the _{DWT coefficient c HL.} Then, it is output to the quantization control unit 1202. In addition, the feature information generation unit 1201 generates _{horizontal edge information using the DWT coefficients b LH} , d _LH , and f _LH , and outputs the horizontal edge information to the quantization control unit 1202. The quantization control unit 1202 generates quantization parameters corresponding to the feature information (brightness, complexity, edge information) from the feature information generation unit 1201 and outputs the quantization parameters to the quantization unit 1203. The quantization unit 1203 uses the quantization parameters input from the quantization control unit 1202 to quantize the DWT coefficients c _LL , c _HL , d _LH , and d _HH held in the horizontal DWT line buffer 1209. The quantized DWT coefficients c _LL , c _HL , d _LH , and d _HH are output from the quantized unit 1203 to the entropy encoding unit 1204.

As described above, the horizontal DWT transform unit 1208 _{outputs the DWT coefficients e LL} , e _HL , f _LH , and f _HH and stores them in the horizontal DWT line buffer 1209. Since the DWT coefficient h _LH corresponding to the lower line has not been calculated, the horizontal edge determination using the _{DWT coefficients e LL} , e _HL , f _LH , and f _{HH cannot be performed yet. The DWT coefficients c LL} , c _HL , d _LH , and d _HH used for quantization and entropy coding _{are obtained by the DWT coefficients e LL} , e _HL , f _LH , and f _HH output by the horizontal DWT transform unit 1208. It will be overwritten. The DWT coefficient d _LH is overwritten and stored in the horizontal edge determination line buffer 1210 (where the DWT coefficient b _LH is stored) before the above overwriting. The image data e of the pixel line buffer Line_Buffer0 is overwritten with the pixel data g after the calculation of the _{DWT coefficient e L.}

After that, the process is repeatedly executed up to the image data of the final line in the same manner.

As can be understood from the above configurations, in particular, the configurations of FIGS. 12 and 13, the quantization parameters cannot be determined until _{the DWT coefficient (for example, h LH) corresponding to the lower line is calculated.} Therefore, it is necessary to hold the _{DWT coefficient (for example, e LL} , e _HL , f _LH , f _HH ) in the horizontal DWT line buffer 1209 until the DWT coefficient corresponding to the lower line is calculated. That is, since a total of two lines of line buffers are required for each subband, the circuit scale of the coding apparatus 100 becomes bloated in the above configuration. An embodiment of the present invention capable of solving the above problems will be described below.

Each embodiment described below is merely an example of a configuration in which the present invention can be realized. Each of the following embodiments can be appropriately modified or modified according to the configuration of the apparatus to which the present invention is applied and various conditions. Therefore, the scope of the present invention is not limited by the configurations described in each of the following embodiments. For example, a configuration in which a plurality of configurations described in the embodiment are combined can be adopted as long as there is no mutual contradiction.

<First Embodiment>
In the above example, the determination of the horizontal edge is performed based on the LH subband coefficient. In the first embodiment of the present invention, the determination of the horizontal edge is executed based on the pixel value (pixel average value). The details are as follows.

As described above with reference to FIG. 3 and the like, the size of each subband of decomposition level 1 is half of the input pixel data in each of the horizontal direction and the vertical direction. The number of each subband coefficient is 1/4 of the number of pixels of the original image. That is, one subband coefficient corresponds to 2 pixels in the horizontal direction × 2 pixels in the vertical direction. Therefore, in the first embodiment of the present invention, the determination of the horizontal edge is executed based on 2 × 2 pixels (2 pixels in the horizontal direction × 2 pixels in the vertical direction) corresponding to one subband coefficient.

The horizontal edge detection method of the first embodiment of the present invention will be described with reference to FIG. FIG. 14A compares the detection method according to the above example (left side) using the subband coefficient with the detection method (right side) of the present embodiment using the 2 × 2 pixel value. The pixel group 1404 including 2 × 2 pixels corresponds to the subband coefficient 1401 at coordinate m-1. Similarly, the pixel group 1405 corresponds to the subband coefficient 1402 of the coordinate m, and the pixel group 1406 corresponds to the subband coefficient 1403 of the coordinate m + 1.

In the present embodiment, the horizontal edge is determined by using the pixel average value of 2 × 2 pixels instead of the subband coefficient in the above example. In the detection process of the present embodiment, when the average value of each pixel exceeds a predetermined threshold value, it is determined to be "large amplitude value", while when it is equal to or less than a predetermined threshold value, it is determined to be "small amplitude value". .. Then, according to the array of the above determination values (“large amplitude value” or “small amplitude value”), it is determined whether or not the horizontal edge is included in the vertical three coefficients.

FIG. 14B illustrates an array of determination values that are determined to include horizontal edges. The vertical coordinate positions (m-1 to m + 1) on the horizontal axis in FIG. 14B correspond to the three pixel groups (m-1 to m + 1) in FIG. 14A. The vertical axis in FIG. 14B corresponds to the pixel average value at each vertical coordinate position. As shown in the figure, the judgment values at the vertical coordinate position are m-1 = small amplitude value, m = large amplitude value, and m + 1 = small amplitude value, respectively, when the vertical coordinate position changes from m-1 to m + 1. There is a change in the amplitude value (pixel average value) across the threshold. The fact that the amplitude value changes across the threshold value as described above indicates that the vertical continuity of the image is broken at the locations corresponding to the three pixel groups, and thus it is determined that the horizontal edge exists. To.

FIG. 15 is a block diagram illustrating the configuration of the coding device 150 according to the present embodiment capable of executing the above-mentioned determination process. The coding device 150 includes a wavelet transform unit 1500, a feature information generation unit 1501, a quantization control unit 1502, a quantization unit 1503, and an entropy coding unit 1504. The above elements correspond to the wavelet transform unit 1200, the feature information generation unit 1201, the quantization control unit 1202, the quantization unit 1203, and the entropy coding unit 1204, respectively, of FIG. It is preferable that the elements (not shown) in FIG. 15, particularly the elements before the plane conversion unit, are configured in the same manner as in FIGS. 1 and 12. Therefore, the coding device 150 of the present embodiment functions as an image pickup device having an image pickup unit.

The wavelet transform unit 1500 includes a pixel line buffer 1505, a vertical DWT transform unit 1506, a vertical DWT line buffer 1507, and a horizontal DWT transform unit 1508 used for DWT conversion of decomposition level 1. Note that unlike the above example, the wavelet transform unit 1500 does not have a horizontal DWT line buffer. The feature information generation unit 1501 has a horizontal edge determination line buffer 1509 used for determining a horizontal edge.

Similar to the above example, in the functional block included in the coding device 150, one or more processors such as a CPU included in the coding device 150 expand the program stored in the non-volatile memory into the volatile memory and execute the program. Realized by.

FIG. 16 is a timing chart showing a time series of DWT transform, feature information generation, and quantization and entropy coding performed in the coding apparatus 150 shown in FIG. In the present embodiment, the coding apparatus 150 executes vertical DWT conversion and horizontal DWT conversion up to decomposition level 1 on the pixel data after plane conversion input in raster order.

FIG. 16 corresponds to FIG. 13 of the above example, but differs in at least the following points. In the bar shown in the “horizontal edge determination line buffer” line of FIG. 16, the pixel average value M for each 2 × 2 pixels output from the pixel line buffer 1505 temporarily enters the horizontal edge determination line buffer 1509 during the corresponding period. Indicates that it is retained. For example, in the periods t4 to t5, the pixel average value M (a + b) for every 2 × 2 pixels for 0.5 lines based on the pixel data a and b is held in the horizontal edge determination line buffer 1509. Further, in the periods t6 to t7, the pixel average value M (c + d) for every 2 × 2 pixels for 0.5 lines based on the pixel data c and d is held in the horizontal edge determination line buffer 1509. In each figure, the pixel average value M is represented by using an overline.

The coding process executed by the coding apparatus 150 shown in FIG. 15 during the period shown in FIG. 16 will be described below in chronological order.

In the periods t0, t1, and t2, the pixel data a, b, and c are input to the pixel line buffer 1505, respectively. The input pixel data a, b, and c are held in the pixel line buffers Line_Buffer0, 1, and 2, respectively, but the vertical DWT transform is not executed.

In the period t3, the vertical DWT transform unit 1506 calculates the DWT coefficients _{b H} using pixel data a in the pixel line buffer 1505, b, or c. Calculated DWT coefficients _{b H} is output to the vertical DWT line buffer 1507 and (Line_Buffer3) and horizontal DWT transform unit 1508. Further, the vertical DWT conversion unit 1506 calculates the DWT coefficient a _L using the pixel data a in the pixel line buffer 1505 and the calculated DWT coefficient b _H, and outputs the DWT coefficient a L to the horizontal DWT conversion unit 1508. The horizontal DWT transform unit 1508 executes horizontal DWT transform on _{the input DWT coefficients a L} and b _{H , calculates the DWT coefficients a LL} , a _HL , b _LH and b _HH, and calculates the feature information generation unit 1501 and the quantum. Output to the conversion unit 1503.

The feature information generation unit 1501 generates _{brightness using the DWT coefficient a LL} , generates complexity using the DWT coefficients a _HL , b _LH , and b _HH, and uses the DWT coefficient a _HL to generate vertical edge information (the first). 1 Feature information) is generated and output to the quantization control unit 1502. The quantization control unit 1502 generates quantization parameters corresponding to the feature information (brightness, complexity, edge information) from the feature information generation unit 1501 and outputs the quantization parameters to the quantization unit 1503. _{The quantization unit 1503 quantizes the DWT coefficients a LL} , a _HL , b _LH , and b _HH input from the horizontal DWT conversion unit 1508 using the quantization parameters input from the quantization control unit 1502, and the entropy code. Output to the conversion unit 1504. Similar to the example of FIG. 6, in the configuration of FIG. 14, the determination of the horizontal edge is not executed when processing the upper end portion or the lower end portion of the image.

The horizontal edge determination line buffer 1509 stores the pixel average value M (a + b) based on the pixel data a and b stored in the pixel line buffer 1505. By holding the determination result acquired by the threshold value determination (FIG. 14) for the pixel average value calculated for each 2 × 2 pixel as 1-bit flag information, storage of a plurality of bits in the line buffer is avoided. You may. The image data a of the pixel line buffer Line_Buffer0 is overwritten with the pixel data d.

In the period t4, the pixel data e is overwritten by the pixel line buffer 1505 (Line_Buffer1) holding the pixel data b. No vertical DWT transform is performed.

In the period t5, the vertical DWT transform unit 1506 calculates the _{DWT coefficient d H} using the pixel data c, d, and e in the pixel line buffer 1505. The calculated DWT coefficient d _H is output to the vertical DWT line buffer 1507 (Line_Buffer4) and the horizontal DWT converter 1508. Further, the vertical DWT conversion unit 1506 calculates the DWT coefficient c _L using the calculated _{DWT coefficient d H} , the pixel data c in the pixel line buffer 1505, and the DWT coefficient b _{H in the vertical DWT line buffer 1507.} Output to the horizontal DWT converter 1508. The horizontal DWT conversion unit 1508 executes horizontal DWT conversion on _{the input DWT coefficients c L} and d _{H , calculates the DWT coefficients c LL} , c _HL , d _LH and d _HH, and calculates the feature information generation unit 1501 and the quantum. Output to the conversion unit 1503.

At the same time, the pixel average value M (a + b) for each 2 × 2 pixels of the pixel data a and b, the pixel average value M (c + d) for each 2 × 2 pixels of the pixel data c and d, and the pixel data e and f2. The pixel average value M (e + f) for each × 2 pixel is output to the feature information generation unit 1501. The pixel average value M (a + b) is held in the horizontal edge determination line buffer 1509. The pixel data c and d are held in the pixel line buffer 1505 (Line_Buffer2, 0). The pixel data e is held in the pixel line buffer 1505 (Line_Buffer1), and the pixel data f is input in the period t5.

Similar to the period t3, the feature information generation unit 1501 generates _{brightness, complexity, and edge information using the DWT coefficients c LL} , c _HL , d _LH , and d _HH , and outputs the lightness, complexity, and edge information to the quantization control unit 1502. In the present embodiment, as shown on the right side of FIG. 17, the horizontal edge information (second feature information) includes the pixel average value M (a + b) of the region 1704, the pixel average value M (c + d) of the region 1705, and the region 1706. It is generated using the pixel average value M (e + f) of. The left side of FIG. 17 contrasts the generation of horizontal edge information using the 1LH subband coefficients 1701 to 1703 in the above examples (FIGS. 12, 13, etc.). The quantization control unit 1502 generates a quantization parameter in the same manner as in the period t3 and outputs the quantization parameter to the quantization unit 1503. _{The quantization unit 1503 quantizes the DWT coefficients c LL} , c _HL , d _LH , and d _HH input from the horizontal DWT conversion unit 1508 using the quantization parameters input from the quantization control unit 1502, and the entropy code. Output to the conversion unit 1504.

The horizontal edge determination line buffer 1509 overwrites and stores the pixel average value M (c + d) based on the pixel data c and d stored in the pixel line buffer 1505 with the pixel average value M (a + b). The image data c of the pixel line buffer Line_Buffer2 is overwritten with the pixel data f.

In the period t6, the pixel data g is overwritten by the pixel line buffer 1505 (Line_Buffer0) holding the pixel data d. No vertical DWT transform is performed.

In the period t7, the vertical DWT transform unit 1506 calculates the DWT coefficients _{f H} with pixel data e in the pixel line buffer 1505, f, and g. Calculated DWT coefficients _{f H} is output to the vertical DWT line buffer 1507 and (Line_Buffer3) and horizontal DWT transform unit 1508. Further, the vertical DWT conversion unit 1506 calculates the DWT coefficient e _L by _{using the calculated DWT coefficient f H} , the pixel data e in the pixel line buffer 1505, and the DWT coefficient d _{H in the vertical DWT line buffer 1507.} Output to the horizontal DWT converter 1508. The horizontal DWT transform unit 1508 executes horizontal DWT transform on _{the input DWT coefficients e L} and f _{H to calculate the DWT coefficients e LL} , e _HL , f _LH and f _HH, and calculates the feature information generation unit 1501 and the quantum. Output to the conversion unit 1503.

At the same time, the pixel average value M (c + d) for each 2 × 2 pixel of the pixel data c and d, the pixel average value M (e + f) for each 2 × 2 pixel of the pixel data e and f, and the pixel data g and h 2 The pixel average value M (g + h) for each × 2 pixel is output to the feature information generation unit 1501. The pixel average value M (c + d) is held in the horizontal edge determination line buffer 1509. The pixel data e and f are held in the pixel line buffer 1505 (Line_Buffer1, 2). The pixel data g is held in the pixel line buffer 1505 (Line_Buffer0), and the pixel data h is input in the period t7.

The feature information generation unit 1501 generates _{brightness, complexity, and edge information using the DWT coefficients e LL} , e _HL , f _LH , and f _HH as in the periods t3 and t5, and outputs the lightness, complexity, and edge information to the quantization control unit 1502. To do. In the present embodiment, as shown on the right side of FIG. 18, the horizontal edge information includes the pixel average value M (c + d) of the area 1804, the pixel average value M (e + f) of the area 1805, and the pixel average value M (e + f) of the area 1806. It is generated using g + h). The left side of FIG. 18 contrasts the generation of horizontal edge information using the 1 LH subband coefficients 1801 to 1803 in the above examples (FIGS. 12, 13, etc.). The quantization control unit 1502 generates a quantization parameter in the same manner as in the periods t3 and t5 and outputs the quantization parameter to the quantization unit 1503. When it is determined that the horizontal edge or the vertical edge exists, the quantization control unit 1502 changes the feature classification as described above with reference to FIG. _{The quantization unit 1503 quantizes the DWT coefficients e LL} , e _HL , f _LH , and f _HH input from the horizontal DWT conversion unit 1508 using the quantization parameters input from the quantization control unit 1502, and the entropy code. Output to the conversion unit 1504.

The horizontal edge determination line buffer 1509 overwrites and stores the pixel average value M (e + f) based on the pixel data e and f stored in the pixel line buffer 1505 with the pixel average value M (c + d). The image data e of the pixel line buffer Line_Buffer1 is overwritten with the pixel data h.

After that, the process is repeatedly executed up to the image data of the final line in the same manner.

According to the configuration of the present embodiment described above, as compared with the configuration described above with reference to FIGS. 12 and 13, efficient coding and suppression of circuit scale in consideration of the characteristics of the image in the vertical direction are achieved. Can be compatible. More specifically, in the configuration of the present embodiment, the horizontal DWT line buffer 1209 for two lines can be reduced, although the pixel line buffer Line_Buffer2 for one line is additionally required as compared with the configuration described above. That is, the total number of line buffers in the configuration of the present embodiment is one line less than the total number of line buffers in the configuration described above. Therefore, in the configuration of the present embodiment, it is possible to generate quantization parameters in consideration of the characteristics of the image in the vertical direction as in the above-described configuration, and the number of line buffers is reduced as compared with the above-mentioned configuration. be able to.

In the configuration of the present embodiment described above, edge determination using pixel values in the vertical direction is executed, but consideration of the characteristics of the image in the vertical direction is not limited as described above. For example, dark area detection may be performed using pixel values over the vertical direction.

Hereinafter, dark portion detection (generation of dark portion information) in the configuration of the present embodiment will be described. First, with reference to FIG. 19, a process of detecting whether or not a dark portion is included in three conversion coefficients (three vertical coefficients) that are continuous in the vertical direction will be described. FIG. 19A shows a low frequency component used for detecting dark areas in this example. The dark part can be detected by executing the detection process using the coefficient of the LL subband of the low frequency component indicating the brightness level. In the detection process of this example, when each coefficient exceeds a predetermined threshold value, it is determined as a "bright part", and when it is equal to or less than a predetermined threshold value, it is determined as a "dark part".

FIG. 19B shows an example in which it is determined to be a dark part. The vertical coordinate positions (m-1 to m + 1) on the horizontal axis in FIG. 19 (b) correspond to the vertical three coefficients (m-1 to m + 1) in FIG. 19 (a). The vertical axis in FIG. 19B corresponds to the coefficient value at each vertical coordinate position. As shown in the figure, the determination values at the vertical coordinate positions are m-1 = dark part, m = bright part, and m + 1 = bright part, respectively. As described above, it is possible to determine whether each coefficient is a bright part or a dark part based on a predetermined threshold value.

However, as described above, the circuit scale becomes bloated in the configuration in which the quantization parameter is determined based on the subband coefficient in the vertical direction. Therefore, in the present embodiment, the dark portion determination is executed using the pixel value instead of the subband coefficient.

A method for detecting a dark portion according to the first embodiment of the present invention will be described with reference to FIG. FIG. 20A compares the detection method according to the above example (left side) using the subband coefficient with the detection method (right side) of the present embodiment using the 2 × 2 pixel value. The pixel group 2004 including 2 × 2 pixels corresponds to the subband coefficient 2001 of the coordinate m-1. Similarly, the pixel group 2005 corresponds to the subband coefficient 2002 of the coordinate m, and the pixel group 2006 corresponds to the subband coefficient 2003 of the coordinate m + 1.

In the present embodiment, the dark portion is determined by using the pixel average value of 2 × 2 pixels instead of the subband coefficient in the above example. In the detection process of the present embodiment, when the average value of each pixel exceeds a predetermined threshold value, it is determined as a "bright area", and when it is equal to or less than a predetermined threshold value, it is determined as a "dark area".

FIG. 20B shows an example of the present embodiment determined to be a dark part. The vertical coordinate positions (m-1 to m + 1) on the horizontal axis in FIG. 20 (b) correspond to the three pixel groups (m-1 to m + 1) in FIG. 20 (a). The vertical axis in FIG. 20B corresponds to the pixel average value at each vertical coordinate position. As shown in the figure, the determination values at the vertical coordinate positions are m-1 = dark part, m = bright part, and m + 1 = bright part, respectively. As described above, it is possible to determine whether the average value of each pixel is a bright part or a dark part based on a predetermined threshold value.

With reference to FIG. 21, feature classification considering the presence of dark areas will be described. When it is determined that a dark part exists in the above-mentioned dark part detection, the feature classification table shown in FIG. 7 is changed as shown in FIG. For example, the quantization control unit 1502 changes the elements classified into the regions Q1 and Q2 (region 2101) so as to be classified into the region Q0 (region 2102). Similarly, the quantization control unit 1502 sets the elements classified into the regions Q4 and Q5 (region 2103) into the region Q3 (region 2004) and the elements classified into the regions Q7 and Q8 (region 2105) into the region Q6 (region Q6). Change so that it is classified as 2106). According to the above configuration, the quantization parameter in the dark region where visual deterioration is easily noticeable can be kept low.

<Second Embodiment>
Hereinafter, the second embodiment of the present invention will be described. In each of the embodiments illustrated below, for elements whose actions and functions are equivalent to those of the premise example or the first embodiment, the reference numerals referred to in the above description will be used and the respective description will be omitted as appropriate.

In the first embodiment, the horizontal edge determination is executed using the pixel average value of 2 × 2 pixels corresponding to one subband coefficient. In the second embodiment, the horizontal edge determination is executed by using the pixel average value of 2 horizontal pixels × 1 vertical pixel (hereinafter, may be referred to as 2 × 1 pixel) in combination.

A method for detecting a horizontal edge according to a second embodiment of the present invention will be described with reference to FIG. 22. FIG. 22A compares the detection method according to the above example (left side) using the subband coefficient with the detection method (right side) of the present embodiment using the 2 × 1 pixel value and the 2 × 2 pixel value. ing. The pixel group 2204 including 2 × 1 pixels corresponds to the subband coefficient 2201 at coordinate m-1. Similarly, the pixel group 2205 including 2 × 2 pixels corresponds to the subband coefficient 2202 of the coordinate m, and the pixel group 2206 including the 2 × 1 pixel corresponds to the subband coefficient 2203 of the coordinate m + 1.

In the present embodiment, the horizontal edge is determined using the pixel average value of 2 × 1 pixel value and the pixel average value of 2 × 2 pixel. In the detection process of the present embodiment, when the average value of each pixel exceeds a predetermined threshold value, it is determined to be "large amplitude value", while when it is equal to or less than a predetermined threshold value, it is determined to be "small amplitude value". .. Then, it is determined whether or not the horizontal edge exists according to the array of the above determination values (“large amplitude value” or “small amplitude value”).

FIG. 22B illustrates an array of determination values that are determined to include horizontal edges. The vertical coordinate positions (m-1 to m + 1) on the horizontal axis in FIG. 22 (b) correspond to the three pixel groups (m-1 to m + 1) in FIG. 22 (a). The vertical axis in FIG. 22B corresponds to the pixel average value of 2 × 1 pixels or 2 × 2 pixels at each vertical coordinate position. As shown in the figure, the judgment values at the vertical coordinate position are m-1 = small amplitude value, m = large amplitude value, and m + 1 = small amplitude value, respectively, when the vertical coordinate position changes from m-1 to m + 1. There is a change in the amplitude value (pixel average value) across the threshold. The fact that the amplitude value changes across the threshold value as described above indicates that the vertical continuity of the image is broken at the locations corresponding to the three pixel groups, and thus it is determined that the horizontal edge exists. To.

FIG. 23 is a block diagram illustrating the configuration of the coding device 230 according to the present embodiment capable of executing the above-mentioned determination process. The coding device 230 includes a wavelet transform unit 2300, a feature information generation unit 2301, a quantization control unit 2302, a quantization unit 2303, and an entropy coding unit 2304. The above elements correspond to the wavelet transform unit 1500, the feature information generation unit 1501, the quantization control unit 1502, the quantization unit 1503, and the entropy coding unit 1504, respectively, of FIG. It is preferable that the elements (not shown) in FIG. 23, particularly the elements before the plane conversion unit, are configured in the same manner as in the first embodiment. Therefore, the coding device 230 of the present embodiment functions as an image pickup device having an image pickup unit.

The wavelet transform unit 2300 includes a pixel line buffer 2305, a vertical DWT transform unit 2306, a vertical DWT line buffer 2307, and a horizontal DWT transform unit 2308 used for DWT conversion of decomposition level 1. Similar to the first embodiment, the wavelet transform unit 2300 does not have a horizontal DWT line buffer. The pixel line buffer 2305 (2 lines) of the second embodiment is one line less than the pixel line buffer 1505 (3 lines) of the first embodiment. The feature information generation unit 2301 has a horizontal edge determination line buffer 2309 used for determining the horizontal edge.

Similar to the first embodiment, the functional block included in the coding device 230 is executed by one or more processors such as the CPU of the coding device 230 expanding the program stored in the non-volatile memory into the volatile memory. It is realized by doing.

FIG. 24 is a timing chart showing a time series of DWT transform, feature information generation, and quantization and entropy coding performed in the coding apparatus 230 shown in FIG. 23. In the present embodiment, the coding device 230 performs the vertical DWT transform and the horizontal DWT transform as in the first embodiment.

FIG. 24 corresponds to FIG. 16 of the first embodiment, but differs in at least the following points. In the bar shown in the “horizontal edge determination line buffer” line of FIG. 24, the pixel average value for each 2 × 1 pixel output from the pixel line buffer 2305 is temporarily held in the horizontal edge determination line buffer 2309 during the corresponding period. Indicates that it has been done. For example, in the periods t3 to t4, the pixel average value M (b) for each 2 × 1 pixel for 0.5 lines based on the pixel data b is held in the horizontal edge determination line buffer 2309. Further, in the periods t5 to t6, the pixel average value M (d) for each 2 × 1 pixel for 0.5 lines based on the pixel data d is held in the horizontal edge determination line buffer 2309.

The coding process executed by the coding apparatus 230 shown in FIG. 23 during the period shown in FIG. 24 will be described below in chronological order.

In the periods t0 and t1, the pixel data a and b are input to the pixel line buffer 2305, respectively. During the above periods t0 and t1, the pixel data of the number of lines required to execute the vertical DWT transform is not input to the wavelet transform unit 2300. Therefore, the input pixel data a and b are held in the pixel line buffers Line_Buffer0 and 1, respectively, but the vertical DWT transform is not executed.

In the period t2, the vertical DWT transform unit 2306 calculates the DWT coefficients _{b H} using pixel data a, b and the input pixel data c in the pixel line buffer 2305. Calculated DWT coefficients _{b H} is output to the vertical DWT line buffer 2307 and (Line_Buffer2) and horizontal DWT transform unit 2308. Further, the vertical DWT conversion unit 2306 calculates the DWT coefficient a _L using the pixel data a in the pixel line buffer 2305 and the calculated DWT coefficient b _H, and outputs the DWT coefficient a L to the horizontal DWT conversion unit 2308. The horizontal DWT transform unit 2308 executes horizontal DWT transform on _{the input DWT coefficients a L} and b _{H , calculates the DWT coefficients a LL} , a _HL , b _LH and b _HH, and calculates the feature information generation unit 2301 and the quantum. Output to the conversion unit 2303.

The feature information generation unit 2301 generates _{brightness using the DWT coefficient a LL} , generates complexity using the DWT coefficients a _HL , b _LH , and b _HH, and uses the DWT coefficient a _HL to generate vertical edge information (the first). 1 Feature information) is generated and output to the quantization control unit 2302. The quantization control unit 2302 generates a quantization parameter corresponding to the feature information from the feature information generation unit 2301 and outputs the quantization parameter to the quantization unit 2303. The quantization unit 2303 uses the quantization parameters input from the quantization control unit 2302 to quantize the DWT coefficients a _LL , a _HL , b _LH , and b _HH input from the horizontal DWT transform unit 2308, and the entropy code. Output to the conversion unit 2304.

The pixel average value M (b) for each 2 × 1 pixel based on the pixel data b is output from the pixel line buffer 2305 (Line_Buffer1) to the horizontal edge determination line buffer 2309 and stored. The image data a of the pixel line buffer Line_Buffer0 is overwritten with the pixel data c after the calculation of the _{DWT coefficient a L.}

In the period t3, the pixel data d is overwritten by the pixel line buffer 2305 (Line_Buffer1) holding the pixel data b. Similar to the periods t0 to t1, the vertical DWT transform is not executed because the pixel data of the number of lines required to execute the vertical DWT transform is not input to the wavelet transform unit 2300.

In the period t4, the vertical DWT transform unit 2306 calculates the DWT coefficients _{d H} using the pixel data c, d and the input pixel data e in the pixel line buffer 2305. The calculated DWT coefficient d _H is output to the vertical DWT line buffer 2307 (Line_Buffer3) and the horizontal DWT converter 2308. Further, the vertical DWT conversion unit 2306 calculates the DWT coefficient c _L using the calculated _{DWT coefficient d H} , the pixel data c in the pixel line buffer 2305, and the DWT coefficient b _{H in the vertical DWT line buffer 2307.} Output to the horizontal DWT conversion unit 2308. The horizontal DWT transform unit 2308 executes horizontal DWT transform on _{the input DWT coefficients c L} and d _{H , calculates the DWT coefficients c LL} , c _HL , d _LH and d _HH, and calculates the feature information generation unit 2301 and the quantum. Output to the conversion unit 2303.

At the same time, the pixel average value M (b) for each 2 × 1 pixel of the pixel data b, the pixel average value M (c + d) for each 2 × 2 pixels of the pixel data c and d, and each 2 × 1 pixel of the pixel data e. The pixel average value M (e) of is output to the feature information generation unit 2301. The pixel average value M (b) is held in the horizontal edge determination line buffer 2309. The pixel data c and d are held in the pixel line buffer 2305 (Line_Buffer0, 1). The pixel data e is input in the period t4.

The feature information generation unit 2301 generates _{brightness, complexity, and edge information using the DWT coefficients c LL} , c _HL , d _LH , and d _HH as in the period t2, and outputs the lightness, complexity, and edge information to the quantization control unit 2302. In the present embodiment, as shown on the right side of FIG. 25, the horizontal edge information (second feature information) includes the pixel average value M (b) of the area 2504, the pixel average value M (c + d) of the area 2505, and the area 2506. It is generated using the pixel average value M (e) of. The left side of FIG. 25 contrasts the generation of horizontal edge information using the 1 LH subband coefficients 2501 to 2503 in the above examples (FIGS. 12, 13, etc.). The quantization control unit 2302 generates a quantization parameter in the same manner as in the period t2 and outputs the quantization parameter to the quantization unit 2303. The quantization unit 2303 uses the quantization parameters input from the quantization control unit 2302 to quantize the DWT coefficients c _LL , c _HL , d _LH , and d _HH input from the horizontal DWT transform unit 2308, and the entropy code. Output to the conversion unit 2304.

The horizontal edge determination line buffer 2309 overwrites the pixel average value M (d) based on the pixel data d stored in the pixel line buffer 2305 with the pixel average value M (b) and stores the pixel average value M (d). The image data c of the pixel line buffer Line_Buffer0 is overwritten with the pixel data e after the calculation of the _{DWT coefficient c L.}

In the period t5, the pixel data f is overwritten by the pixel line buffer 2305 (Line_Buffer1) holding the pixel data d. In the period t5, as in the periods t0, t1, and t3, the vertical DWT conversion is not executed because the pixel data of the number of lines required to execute the vertical DWT conversion is not input to the wavelet transform unit 2300.

In the period t6, the vertical DWT transform unit 2306 calculates the DWT coefficients _{f H} with the pixel data e, f and input pixel data g in the pixel line buffer 2305. Calculated DWT coefficients _{f H} is output to the vertical DWT line buffer 2307 and (Line_Buffer2) and horizontal DWT transform unit 2308. Further, the vertical DWT conversion unit 2306 calculates the DWT coefficient e _L by _{using the calculated DWT coefficient f H} , the pixel data e in the pixel line buffer 2305, and the DWT coefficient d _{H in the vertical DWT line buffer 2307.} Output to the horizontal DWT conversion unit 2308. The horizontal DWT transform unit 2308 executes horizontal DWT transform on _{the input DWT coefficients e L} and f _{H to calculate the DWT coefficients e LL} , e _HL , f _LH and f _HH, and calculates the feature information generation unit 2301 and the quantum. Output to the conversion unit 2303.

At the same time, the pixel average value M (d) for each 2 × 1 pixel of the pixel data d, the pixel average value M (e + f) for each 2 × 2 pixel of the pixel data e, f, and each 2 × 1 pixel of the pixel data g. The pixel average value M (g) of is output to the feature information generation unit 2301. The pixel average value M (d) is held in the horizontal edge determination line buffer 2309. The pixel data e and f are held in the pixel line buffer 2305 (Line_Buffer0, 1). The pixel data g is input in the period t6.

The feature information generation unit 2301 generates _{brightness, complexity, and edge information using the DWT coefficients e LL} , e _HL , f _LH , and f _HH as in the periods t2 and t4, and outputs the lightness, complexity, and edge information to the quantization control unit 2302. To do. In the present embodiment, as shown on the right side of FIG. 26, the horizontal edge information includes the pixel average value M (d) of the area 2604, the pixel average value M (e + f) of the area 2605, and the pixel average value M (e + f) of the area 2606. It is produced using g). The left side of FIG. 26 contrasts the generation of horizontal edge information using the 1 LH subband coefficients 2601 to 2603 in the above examples (FIGS. 12, 13, etc.). The quantization control unit 2302 generates a quantization parameter in the same manner as in the periods t2 and t4 and outputs the quantization parameter to the quantization unit 2303. The quantization unit 2303 uses the quantization parameters input from the quantization control unit 2302 to quantize the DWT coefficients e _LL , e _HL , f _LH , and f _HH input from the horizontal DWT conversion unit 2308, and the entropy code. It is output to the quantization unit 2304.

The horizontal edge determination line buffer 2309 overwrites the pixel average value M (f) based on the pixel data f stored in the pixel line buffer 2305 with the pixel average value M (d) and stores the pixel average value M (f). The image data e of the pixel line buffer Line_Buffer0 is overwritten with the pixel data g after the calculation of the _{DWT coefficient e L.}

After that, the process is repeatedly executed up to the image data of the final line in the same manner.

According to the configuration of the present embodiment described above, the same technical effect as that of the first embodiment is achieved. In addition, the pixel line buffer for one line can be further reduced as compared with the first embodiment.

<Other Embodiments>
Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications and changes can be made within the scope of the gist thereof.

In the above embodiment, the coding devices 100, 150, and 230 are configured as an image pickup device having the image pickup unit 101, but the PC does not have the image pickup section as described above and the RAW data is supplied from the outside. An information processing device such as the above may function as a coding device according to the present invention. That is, the imaging unit 101 and the recording medium 108 in the above embodiments are not essential components for the coding apparatus according to the present invention.

Further, for example, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or a device via a network or a storage medium, and one or more processors of the computer of the system or the device program the program. It can also be realized by the process of reading and executing. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100, 150, 230 Coding device 103, 1200, 1500, 2300 Wavelet transform unit (conversion means)
104, 1201, 1501, 2301 Feature information generation unit (feature information generation means)
105, 1202, 1502, 2302 Quantization control unit (quantization control means)
106, 1203, 1503, 2303 Quantization unit (quantization means)
107, 1204, 1504, 2304 Entropy encoding unit 1205, 1505, 2305 Pixel line buffer (buffer means)
1207, 1507, 2307 Vertical DWT line buffer (buffer means)

Claims (12)

A buffer means that temporarily holds the input pixel data,
A conversion means that executes wavelet transform on the pixel data to generate a plurality of subband coefficients, and
A quantization means for quantizing the subband coefficient according to a quantization parameter, and
A quantization control means for determining the quantization parameter is provided.
The quantization control means corresponds to the pixel data held in the buffer means and corresponding to the subband coefficient quantized by the quantization means and the subband coefficient located in the direction perpendicular to the subband coefficient. A coding device characterized in that the quantization parameter is determined based on pixel data. The quantization control means corresponds to the first feature information based on the subband coefficient quantized by the quantization means and the subband coefficient held in the buffer means and quantized by the quantization means. The claim is characterized in that the quantization parameter is determined based on the second feature information based on the pixel data to be generated and the pixel data corresponding to the subband coefficient located in the direction perpendicular to the subband coefficient. The coding apparatus according to 1. The buffer means includes the pixel data corresponding to the subband coefficient quantized by the quantization means and pixel data of a plurality of lines corresponding to the subband coefficient located in the direction perpendicular to the subband coefficient. The coding device according to claim 2, wherein one line of pixel data is temporarily held. The coding device according to claim 2 or 3, wherein the buffer means holds a determination result based on a predetermined threshold value with respect to the pixel data. Any one of claims 2 to 4, wherein the quantization control means generates the quantization parameter based only on the first feature information at the upper end portion or the lower end portion of the image. The encoding device according to the section. The coding device according to any one of claims 2 to 5, wherein the feature information includes edge information indicating the existence of an edge. The sixth aspect of claim 6 is characterized in that the quantization control means sets the value of the quantization parameter smaller when the edge information indicates the existence of the edge than when the edge does not exist. Encoding device. The coding device according to any one of claims 2 to 7, wherein the feature information includes dark portion information indicating the existence of a dark region. 8. The quantization control means is characterized in that when the dark area information indicates the existence of the dark area, the value of the quantization parameter is set smaller than that when the dark area does not exist. The encoding device according to. The coding device according to any one of claims 1 to 9,
An image pickup apparatus comprising: an image pickup means including a lens optical system and an image pickup element. A control method for a coding device including a buffer means for temporarily holding input pixel data.
Performing wavelet transform on the pixel data to generate multiple subband coefficients,
Quantizing the subband coefficient according to the quantization parameters,
The quantization parameter is set based on the pixel data corresponding to the quantized subband coefficient held in the buffer means and the pixel data corresponding to the subband coefficient located in the direction perpendicular to the subband coefficient. A control method characterized by determining. A program for causing a computer to function as each means of the coding apparatus according to any one of claims 1 to 9.

Images