JP2009290556A - Image information processing apparatus and image information processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image information processing apparatus and an image information processing method by which quantization processing with reduced deterioration of image quality can be performed and data processing can be performed at high speed. <P>SOLUTION: The apparatus includes a blocking section 132 which divides pixel data outputted from an imaging element according to a predetermined number to generate blocks, a division processing section 134 which divides the blocks into subblocks according to a level change of the pixel data in the respective blocks, a quantization section 136 which quantizes the pixel data by a quantization word length determined according to a change amount of a pixel data level for each subblock thus generated and a section length of each subblock, and generates quantization data, and a compression data generating section 138 which aggregates the information on the level change of the pixel data in the blocks and the quantization data and generates compression data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image information processing apparatus and an image information processing method.

  An imaging device such as a digital camera or a video camera converts a subject into digital data using an imaging device such as a CCD or CMOS. In recent years, the number of pixels of an image sensor has been increased, and an image pickup apparatus is required to have high performance. As the number of pixels of the image sensor increases, the processing load of the image signal in the image capturing apparatus increases. Therefore, the imaging apparatus is required to process data at high speed so that operability is smooth for the user.

  In addition, a signal output from an image sensor such as a CCD or CMOS may be recorded on a recording medium after being subjected to irreversible compression processing or signal processing, but is recorded as RAW data on the recording medium without performing signal processing. In some cases. In this case, the amount of data transmitted to the internal bus of the imaging apparatus is large, and the time required for the writing process to the recording medium and the reading process from the recording medium becomes longer. For example, Patent Documents 1 and 2 disclose a technique for processing RAW data in an imaging apparatus.

JP-A-5-191770 JP 2007-228515 A

  For example, Patent Document 1 discloses a technique for reversibly compressing and transmitting RAW data during data transmission on an internal bus of an imaging apparatus. However, even if lossless compression is performed, for example, Huffman coding or the like is variable length coding processing, so it is difficult to maintain a constant bandwidth on the internal bus. Further, the variable length coding process has a problem that the effect of bandwidth reduction cannot always be obtained.

  In Patent Document 2, image data read / written on a recording medium is compressed. In this compression processing, the maximum and minimum values of pixel data in a block made up of a predetermined number of pixel data, position information in the block, and a subtraction value obtained by subtracting the minimum value from the pixel data in the block are quantized. The The compression process of Patent Document 2 is a fixed-length encoding process, and can be achieved by quantization of 7 bits per pixel when the compression efficiency is 8 bits / pix.

  However, in the compression process, simply reducing the quantization word length to 4 bits or the like to further improve the compression efficiency means that if there is a part where the waveform fluctuates significantly in the block, quantization noise becomes obvious. There was a problem. For example, if the subject is a person standing against a black curtain and a boundary portion having a gradation change between a dark background and a bright person's skin color is present in one block to be compressed, the skin color portion Quantization noise such as a step becomes conspicuous.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to enable quantization processing with reduced image quality degradation and to further speed up data processing. Another object of the present invention is to provide a new and improved image information processing apparatus and image information processing method.

  In order to solve the above-described problem, according to an aspect of the present invention, a block forming unit that generates a block by dividing pixel data output from an image sensor every predetermined number, and changes in the level of pixel data in the block Pixel data with a quantization word length determined according to the amount of change in the pixel data level and the length of the sub-block section for each generated sub-block An image information processing apparatus is provided that includes a quantization unit that quantizes data and generates quantized data, and a compressed data generation unit that aggregates quantized data and information related to level changes of pixel data in a block to generate compressed data The

  The division processing unit may divide the block and generate sub-blocks according to the amplitude value and change amount of the pixel data level in the block.

  The division processing unit includes a division determination unit that determines the number and division position for dividing a block based on the amplitude value and change amount of the pixel data level in the block, and a block according to the determined division number and division position. And a sub-block generating unit that generates sub-blocks.

  In order to solve the above-described problem, according to another aspect of the present invention, from compressed data, information on a level change of pixel data in a block made up of a predetermined number of pixel data, and a sub-block into which the block is divided Based on information on level change, an extraction unit that extracts quantized data in which pixel data is quantized with a quantization word length determined according to the amount of change in pixel data level and the length of the sub-block interval Thus, there is provided an image information processing apparatus having an inverse quantization unit that inversely quantizes quantized data and generates inversely quantized pixel data.

  The information related to the level change may be information related to the amplitude value and the change amount of the level of the pixel data.

  In order to solve the above-described problem, according to another aspect of the present invention, a step of generating a block by dividing pixel data output from an image sensor into a predetermined number, and a level change of pixel data in the block The block is divided according to the step of generating the sub-block, and the pixel data is converted with the quantization word length determined according to the change amount of the pixel data level and the length of the sub-block section for each generated sub-block. There is provided an image information processing method including a step of quantizing and generating quantized data, and a step of aggregating information on level change of pixel data in a block and quantized data to generate compressed data.

  In order to solve the above-described problem, according to another aspect of the present invention, from compressed data, information on a level change of pixel data in a block made up of a predetermined number of pixel data, and a sub-block into which the block is divided Extracting quantized data in which pixel data is quantized with a quantized word length determined according to the amount of change in pixel data level and the length of a sub-block interval for each, and based on information on level change An image information processing method comprising: dequantizing the quantized data to generate dequantized pixel data.

  According to the present invention, quantization processing with reduced image quality degradation can be performed, and further, data processing speed can be increased.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  First, the imaging device 100 according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram illustrating an imaging apparatus 100 according to the present embodiment. The imaging apparatus 100 according to this embodiment is, for example, a digital still camera, a digital video camera, or the like, and can record a subject image as digital data.

As shown in FIG. 1, the imaging apparatus 100 includes, for example, an imaging element 101, an AFE circuit 102, a digital image processing unit 103, an SDRAM 104, a ROM 105, a storage device 106, and the like.
The digital image processing unit 103 includes a camera signal preprocessing unit 111, a camera signal processing unit 112, a resolution conversion unit 113, a JPEG engine 114, a CPU 115, a video output encoder 116, an SDRAM controller 117, and the like. Each of these components is connected to each other by an internal bus 118. Furthermore, the digital image processing unit 103 according to the present embodiment includes a RAW compression unit 121 and a RAW expansion unit 122.

  The imaging device 101 is a solid-state imaging device such as a CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor) imaging device, and converts light incident from a subject through an unillustrated lens block (optical system) into an electrical signal. To do.

  The AFE circuit 102 performs sample hold on the image signal output from the image sensor 101 so as to maintain a good S / N (Signal / Noise) ratio by CDS processing. The AFE circuit 102 controls the gain by AGC processing, performs A / D (Analog / Digital) conversion, and outputs a digital image signal.

  The digital image processing unit 103 is formed as a SoC (System on a Chip) circuit, for example. In the digital image processing unit 103, the camera signal pre-processing unit 111 performs a signal correction process for a defective pixel in the image sensor 101, a shading process for correcting a decrease in the amount of peripheral light of the lens, and the like on the image signal supplied from the AFE circuit 102. Apply. Further, the camera signal preprocessing unit 111 outputs the processed signal as RAW data.

The RAW compression unit 121 compresses the RAW data from the camera signal preprocessing unit 111 by a compression method described later, and supplies the compressed RAW data to the SDRAM 104 via the SDRAM controller 117.
The RAW decompression unit 122 decompresses the compressed RAW data read from the SDRAM 104 via the SDRAM controller 117 by a method described later and outputs the decompressed data to the camera signal processing unit 112.

  The camera signal processing unit 112 performs demosaic processing on the RAW data from the RAW expansion unit 122. Thereafter, the camera signal processing unit 112 performs so-called camera signal processing such as signal detection processing for AF (Auto Focus), AE (Auto Exposure), white balance control, and signal correction processing represented by white balance adjustment, Or a part of the process is executed. Furthermore, the camera signal processing unit 112 converts the image data after signal correction into a luminance signal (Y) and color difference signals (RY, BY) in a predetermined format such as 4: 2: 2.

  The resolution conversion unit 113 receives image data processed by the camera signal processing unit 112 or image data decompressed and decoded by the JPEG engine 114 and converts the image data into a predetermined resolution.

  The JPEG engine 114 compresses and encodes the image data processed by the resolution converter 113 to generate JPEG encoded data. Further, the JPEG engine 114 decompresses and decodes JPEG image data read from the storage device 106. The digital image processing unit 103 may be provided with an encoding / decoding engine other than the JPEG engine 114 using a still image compression method or a moving image compression method.

  The CPU 115 controls the entire digital image processing unit 103 and the entire imaging apparatus 100 by executing a program stored in the ROM 105, and executes various calculations for the control.

  The video output encoder 116 is, for example, an NTSC (National Television Standards Committee) encoder. The video output encoder 116 generates an image signal for monitor display based on the image data output from the resolution conversion unit 113 and the like, and outputs the image signal to a monitor (not shown) or the video output terminal 26a.

  The SDRAM controller 117 is an interface block for the SDRAM 104 and includes an address decoder and the like. The SDRAM controller 117 controls writing and reading operations of the SDRAM 104 according to a control signal from the CPU 115.

  The SDRAM 104 is a volatile memory such as a DDR (Double Data Rate) SDRAM. The SDRAM 104 is a work area for data processing in the digital image processing unit 103. The SDRAM 104 includes a capture data area 104a, a JPEG code area 104b, a CPU work area 104c, and the like. The capture data area 104 a temporarily stores data captured from the image sensor 101, that is, RAW data compressed by the RAW compression unit 121. The JPEG code area 104b temporarily stores image data encoded by the JPEG engine 114, data used in the encoding / decoding process, and the like. The CPU work area 104c temporarily stores data used in the processing of the CPU 115.

  The ROM 105 holds programs executed by the CPU 115 and various data. The ROM 105 is a nonvolatile memory such as an EEPROM (Electronically Erasable and Programmable ROM) or a flash memory.

  The storage device 106 is a device for recording an encoded image data file. The storage device 106 includes, for example, a recording medium such as a flash memory, an optical disk, and a magnetic tape, and a recording / reproducing drive for the recording medium.

  In the imaging apparatus 100 as described above, an imaging signal from the imaging element 101 is sequentially supplied to the AFE circuit 102, subjected to CDS processing and AGC processing, converted into a digital signal, and the camera of the digital image processing unit 103. The signal is supplied to the signal preprocessing unit 111. The camera signal pre-processing unit 111 generates RAW data in which defective pixel correction or shading correction is performed on the input image signal. This RAW data is compressed by the RAW compression unit 121 and then stored in the SDRAM 104. Once written.

  When the RAW data is read from the SDRAM 104, it is expanded by the RAW expansion unit 122 and then subjected to various image quality correction processes by the camera signal processing unit 112. The processed image data is temporarily stored in the SDRAM 104, for example, and then converted into data having a resolution suitable for display by the resolution converter 113, and further stored in the SDRAM 104, for example, and then supplied to the video output encoder 116. . Thereby, a camera through image is displayed on the monitor.

  In addition, when image recording is requested through an input unit (not shown), the resolution conversion unit 113 converts the image data processed by the camera signal processing unit 112 into data having a resolution set for recording as necessary. For example, the data is converted and temporarily stored in the SDRAM 104. The JPEG engine 114 compresses and encodes image data to generate encoded data. For example, the encoded data is once recorded in the SDRAM 104 and then recorded in the storage device 106.

  The image data (encoded data) recorded in the storage device 106 is decompressed and decoded by the JPEG engine 114 and the resolution is converted by the resolution converter 113. Thereafter, the image data is output to the video output encoder 116 so that the image can be displayed on the monitor.

  The digital image processing unit 103 according to the present embodiment is provided with a RAW compression unit 121 that compresses RAW data at the input position of image data from the camera signal preprocessing unit 111 to the internal bus 118. Thereby, the amount of RAW data transmitted to the SDRAM 104 through the internal bus 118 can be reduced. In addition, a RAW expansion unit 122 that expands RAW data is provided at an input position of image data from the internal bus 118 to the camera signal processing unit 112. As a result, the amount of RAW data transmitted from the SDRAM 104 to the camera signal processing unit 112 can be reduced.

  According to the present embodiment, the transmission load on the internal bus 118 during the imaging operation can be reduced, and the time required for the write / read processing for the SDRAM 104 can be shortened. In particular, by simplifying the compression / decompression process as much as possible, the effect of shortening the processing time can be increased. In addition, power consumption can be suppressed by reducing the transmission frequency on the bus.

  Further, according to the present embodiment, the capacity of the SDRAM 104 can be reduced. In addition, use of the SDRAM 104 area for other processing, storing RAW data for multiple frames, increasing the number of images that can be shot continuously, or improving the shooting speed, etc. Contribute to. Therefore, it is possible to realize a high-performance, small, and low-cost imaging apparatus 100 that can shorten the time required for imaging processing and data recording processing.

  In the RAW data compression processing by the RAW compression unit 121, the quality of the RAW data can be completely retained by using a lossless compression method. If it becomes a level that cannot be detected with the naked eye when converted to a color difference signal, the image quality is acceptable. Generally, if a PSNR (Peak Signal to Noise Ratio) at the time of conversion to a luminance / color difference signal is about 50 dB to 40 dB, the compression distortion is at an acceptable level.

  Furthermore, when the RAW data is compressed, the imaging apparatus 100 according to the present embodiment can perform encoding with a fixed length. As a result, the bandwidth of RAW data read / written to / from the SDRAM 104 can be kept constant, and the transmission load on the internal bus 118 can be stably reduced. In addition, handling of RAW data in the camera signal processing unit 112 (for example, reading control processing from the SDRAM 104) and transmission control processing of RAW data through the internal bus 118 can be simplified.

  For example, in the case of encoding with variable length, when reading the compressed RAW data from the SDRAM 104, it is often necessary to access in burst form. On the other hand, by encoding with a fixed length, the address on the SDRAM 104 of the RAW data at an arbitrary position can be easily calculated and read out. Therefore, the camera signal processing unit 112 may process the entire screen partially (for example, in the form of a strip in the vertical direction) with only a delay line that is about a fraction of 1H. Further, it can be accessed by a DMA (Direct Memory Access) controller in the digital image processing unit 103.

  Below, the structure of the RAW compression part 121 of the imaging device 100 of this embodiment is demonstrated in detail. The imaging apparatus 100 according to the present embodiment uses a lossy compression / decompression method that can perform fixed-length encoding, maintain good image quality, and can be realized with relatively simple processing.

  In the following example, the raw data signal is 14 bits per pixel. For example, as shown in FIG. 6, 20 pixels in the horizontal direction having the same color component are converted into encoded data of one block. FIG. 6 is a schematic diagram showing a Bayer array and one extracted same color line. The quantization word length is variable, for example, 2 to 4 bits. The RAW compression unit 121 of this embodiment takes 20 pixels as one block, and compresses this one block to a fixed length of 128 bits. As a result, there are 6.4 bits per pixel. Normally, if 14-bit data has 20 pixels as one block, the amount of data corresponding to one block is 280 bits when RAW data is not compressed. On the other hand, according to the present embodiment, the data amount of 280 bits can be compressed to 128 bits, and a compression rate that reduces the data amount to about 46% can be achieved.

FIG. 2 is a block diagram showing the RAW compression unit 121 according to the present embodiment.
As shown in FIG. 2, the RAW compressing unit 121 includes a polygonal line compressing unit 131, a blocking unit 132, a buffer 133, a block division number determining unit 134, a subtracting unit 135, a quantizing unit 136, a quantized data buffer 137, and a packing unit. 138 and the like.

  The polygonal line compression unit 131 nonlinearly compresses the input 14-bit RAW data into 10-bit data by approximation using a polygonal line. The polygonal line compression unit 131 is provided for the purpose of improving the overall compression efficiency by reducing the gradation as much as possible before the subsequent compression procedure. Depending on the target compression rate, it may be omitted. In this case, the reverse broken line extension unit 192 provided at the output stage of the RAW extension unit 122 described later with reference to FIG. 11 can also be omitted.

Here, FIG. 9 is a graph showing an example of a broken line used in the broken line compression unit 131.
The gradation of the input data is converted by the graph shown in FIG. In this example, in accordance with human visual characteristics, a higher gradation is assigned as input data is smaller, that is, darker (or lighter in color). Such a polygonal line may be prepared for each color component, for example, and used by switching for each color component of the input pixel.

  In the polygonal line compression unit 131, for example, after converting the gradation of the input data using such a polygonal line, the converted data is divided by 16 (that is, shifted downward by 4 bits) to become 10-bit data. Compress. At this time, the discarded lower bits are rounded off, for example. A ROM table that stores the input data and the compressed output data in association with each other based on the above calculation is prepared, and the polygonal line compression unit 131 converts the input / output data according to the ROM table. Also good.

  The blocking unit 132 divides the data output from the polygonal line compression unit 131 into blocks composed of the same color component pixels for 20 pixels adjacent in the horizontal direction, and outputs the divided blocks. Thereby, the correlation of the data in a block becomes strong, and the image quality deterioration by subsequent quantization processing can be suppressed.

For example, when a Bayer array image sensor as shown in FIG. 6 is used, in the output data, the repetition of the R component and the Gr component and the repetition of the B component and the Gb component appear for each line. For example, when the R component and the Gr component repeatedly appear in the input data to the blocking unit 132 as in R ₀ , Gr ₀ , R ₁ , Gr ₁ ,..., R ₁₉ , Gr ₁₉ , R ₀ , R ₁ , R ₂ ,..., R ₁₉ , Gr ₀ , Gr ₁ ,..., Gr ₁₉ , and the output order is changed so that 20 pixels of the same color component appear continuously. To block.

  The buffer 133 holds data from the blocking unit 132. The buffer 133 holds the block corresponding to the input data and adjusts the delay time until the minimum value is calculated by the block division number determination unit 134.

  The block division number determination unit 134 determines the division number for further dividing the block into sub-blocks and compression elements. Details of the block division number determination unit 134 will be described later. The block division number determination unit 134 is an example of a division processing unit.

  The subtracting unit 135 subtracts the minimum value in the corresponding block output from the block division number determining unit 134 from the pixel data output from the blocking unit 132. This subtraction is equivalent to subtracting the DC offset common to the pixels in one block from the data of each pixel.

  The quantization unit 136 quantizes the output data of the subtraction unit 135 according to the shift amount output from the block division number determination unit 134. In this embodiment, as an example, quantization is performed with a variable length of 2 to 5 bits.

For example, the quantization unit 136 divides the output data of the subtraction unit 135 by the shift amount using an integer type divider. Also, when the quantization step is limited to a power of 2, a right shifter can be applied, thereby reducing the circuit scale. If the right shifter is used on the compression processing side, the circuit scale can be similarly reduced in the inverse quantization on the decompression processing side.
The quantized data buffer 137 temporarily holds the quantized data for 20 pixels output from the quantizing unit 136.

  The packing unit 138 uses the output data from the quantized data buffer 137 and the block division number determination unit 134 to pack into compressed data of 128 bits per block. The output data from the block division number determination unit 134 includes a compression type that indicates the minimum value of pixel data in the block, the shift amount, the coordinates (position information) of sub-blocks, 20 pixels of quantized data, and in which format. Are packed as shown in FIG. As a result, the data amount of 280 bits can be compressed to 128 bits. The packing unit 138 is an example of a compressed data generation unit.

  Next, the block division number determination unit 134 of this embodiment will be described with reference to FIGS. FIG. 3 is a block diagram showing the block division number determination unit 134 according to the present embodiment. The block division number determination unit 134 includes, for example, a buffer 141, a block division determination unit 142, and a block division data generation unit 143.

  The buffer 141 holds data from the blocking unit 132. The buffer 141 holds the block corresponding to the input data and adjusts the delay time until the block division determination unit 142 outputs the coordinates of the subblock, the block division determination end signal, and the like.

  As illustrated in FIG. 4, the block division determination unit 142 includes, for example, a maximum value / minimum value detection unit 151, a difference calculation unit 152, a word length detection unit 153, a region detection unit 154, an address counter 155, a change point latch unit 156, It includes a change point counter 157, a type determination unit 158, and a region binarization unit 159. The block division number determination unit 134 determines whether to generate a subblock or not to generate a subblock, and generates subblock division coordinates. FIG. 4 is a block diagram illustrating the block division determination unit 142 according to the present embodiment. The block division determination unit 142 is an example of a division determination unit.

  The maximum value / minimum value detection unit 151 detects the maximum value (MAX) and the minimum value (MIN) from the data of the horizontal 20 pixels that are formed into blocks for each block. The maximum value / minimum value detection unit 151 outputs the detected maximum value (MAX) and minimum value (MIN) to the difference calculation unit 152 and the region detection unit 154.

  The difference calculation unit 152 calculates the difference between the maximum value (MAX) and the minimum value (MIN) in one block and calculates the dynamic range (DR0) in one block. The difference calculation unit 152 outputs the dynamic range (DR0) to the word length detection unit 153.

  The word length detection unit 153 detects the number of digits of the dynamic range of the pixel data existing in one block from the dynamic range (DR0). The word length detector 153 outputs the number of detected dynamic range digits (DEPTH).

  The region detection unit 154 determines a transition section (transition region) until the input waveform of the pixel data swings between the maximum and minimum ends based on the maximum value (MAX) and the minimum value (MIN) and the following threshold value. A pixel in the transition area is set to AREA_DET = 1, a pixel in the vicinity of the minimum value is set to AREA_DET = 0, and a pixel in the vicinity of the maximum value is set to AREA_DET = 2 and outputs AREA_DET. FIG. 7 shows the relationship between pixel data input values (Pi) and regions. FIG. 7 is an explanatory diagram showing an input waveform of pixel data.

The threshold value is as follows, for example.
MIN ≦ Pi <(MAX−MIN) / 4 + MIN... AREA_DET = 0
(MAX−MIN) / 4 + MIN ≦ Pi <3 × (MAX−MIN) / 4 + MIN... AREA_DET = 1
3 × (MAX−MIN) / 4 + MIN ≦ Pi ≦ MAX... AREA_DET = 2

  Note that this threshold value is an example. For example, a waveform trend may be reflected on the threshold value based on a histogram or statistical information of the input waveform. Although the threshold value is two, it is not limited to this. The region detection unit 154 outputs AREA_DET to the change point latch unit 156 and the region binarization unit 159.

  The address counter 155 counts up (counts up) the input pixel signals of 20 pixels. The address counter 155 outputs a block division determination end signal to the subsequent block division data generation unit 143 every time 20 pixels are input, and resets itself to 0 (zero).

  The change point latch unit 156 detects the change point of the region from the 2-bit output of AREA_DET output from the region detection unit 154. Then, the change point latch unit 156 has a maximum of three area change points (POSn (n = 0, 1, 2)) as sub-block switching coordinate candidates (division coordinate candidates when the block is divided into sub-blocks). Hold up. The change point latch unit 156 holds the counter value (0 to 19 in this embodiment) output from the address counter 155 of the detected change point, and uses the counter value as a coordinate candidate. The change point latch unit 156 outputs a counter value (POSn (n = 0, 1, 2) = 0 to 19) of the change point.

  The change point counter 157 counts up the pixel signal every time the change point latch unit 156 holds the value of the change point counter as a coordinate candidate. Then, the change point counter 157 counts and holds the number of sub-block switching coordinate candidates. The change point counter 157 outputs the number of switching coordinate candidates (CNT1).

  The type determination unit 158 determines to generate a sub block or not to generate a sub block. The decision to generate a sub-block, ie to divide one block into sub-blocks, is based on the amplitude of the level of pixel data in one block and the number of switching coordinate candidates. For example, the number of dynamic range digits (DEPTH) detected by the word length detection unit 153 is the same as the number of digits of the input signal (in this embodiment, 10 digits) or less than 1/2. When it is within the range (for example, DEPTH = 10 or 9), the change in pixel data in one block is large (the amplitude is large).

Type determination unit 158 determines that a sub-block is generated when the amplitude in one block is large and one block can be divided into four or less sub-blocks. In other cases, type determination unit 158 determines that no sub-block is generated, and compresses pixel data by taking one block as a group. The type is determined as follows. The type is included in the compressed data because it is necessary to determine the procedure for decompression. The case where one block can be divided into four or less sub-blocks is when the value output from the change point counter 157 is CNT1 ≦ 3.
type 00: when only one block is used type 01: when two sub-blocks are generated type 10: when three sub-blocks are generated type 11: when four sub-blocks are generated The type determination unit 158 determines the determined type. Output.

The area binarization unit 159 holds AREA_DET values for each of up to four sub-blocks. Then, the area binarization unit 159 binarizes the AREA_DET value for each sub block according to the transition area, the maximum value, or the minimum value vicinity area.
Specifically, when AREA_DET = 0, ARAn (n = 0, 1, 2, 3) = 1, and when AREA_DET = 1, ARAn (n = 0, 1, 2, 3) = 0 and AREA_DET. When = 2, AREAn (n = 0, 1, 2, 3) = 1. The area binarization unit 159 outputs ARAn (n = 0, 1, 2, 3) = 0 or 1. AREAn (n = 0, 1, 2, 3) means each sub-block of up to four sub-blocks.

Next, the block division data generation unit 143 will be described with reference to FIG. FIG. 5 is a block diagram showing the block division data generation unit 143 according to this embodiment.
As illustrated in FIG. 5, the block division data generation unit 143 includes, for example, an address counter 161, a change point holding unit 162, a latch pulse output unit 163, a section width calculation unit 164, a maximum / minimum value detection unit 165, and a difference calculation. Unit 166, minimum value latch unit 167, selector 168, region binarized value holding unit 169, quantization word length determination unit 170, shift amount calculation unit 171, shift amount latch unit 172, selector 173, change point holding unit 174, It consists of an address counter 175.
The block division data generation unit 143 generates elements necessary for quantization in subblock units. The block division data generation unit 143 is an example of a sub-block generation unit.

Both the address counter 161 and the address counter 175 count the number of pixels for one block.
The address counter 161 receives the block division determination end signal and sets it as an address count start signal. The output (CNT2) of the address counter 161 is output via the output (latch2) of the latch pulse output unit 163, the maximum value / minimum value detection unit 165, the minimum value latch unit 167 for determining the minimum value for each subblock, and the shift amount. The shift amount latch unit 172 to be determined is controlled.

  The address counter 175 is activated when the address counter 161 finishes counting for 20 pixels. The address counter 175 outputs a CONT signal. The CONT signal controls to switch the selector 173 based on the number of divisions according to the type (type00, 01, 10, 11) and the coordinate registers (POS0 ', 1', 2 '). By this control, pixel data is quantized for each sub-block.

  The change point holding unit 162 and the change point holding unit 174 hold a maximum of three coordinates according to the number of sub-blocks. The change point holding unit 162 holds a change point counter value (coordinate register POSn (n = 0, 1, 2) = 0 to 19). The change point holding unit 174 holds a change point counter value (coordinate registers POSn (n = 0 ′, 1 ′, 2 ′) = 0 to 19). The coordinate registers POS0 'to POS2' are copies of the coordinate registers POS0 to POS2, and are held until they are used when packed as compressed data by the packing unit 138 after the quantization.

  The latch pulse output unit 163 outputs a latch pulse (latch2) in units of sub-blocks based on the counter value from the address counter 161 and the coordinate register from the change point holding unit 162.

The section width calculation unit 164 detects the number of sub-block divisions based on the type (type00, 01, 10, 00). The section width calculation unit 164 calculates a section width from the coordinate difference. The section width calculation unit 164 calculates, for example, as follows and outputs the section widths W0, W1, W2, and W3.
In the case of type 00, the section width W0 = 19. The section widths W1, W2, and W3 are not used.
In the case of type01, the section widths W0 = POS0-0 and W1 = 19-POS0. The section widths W2 and W3 are not used.
In the case of type 10, the section width is W0 = POS0-0, W1 = POS1-POS0, and W2 = 19-POS1. The section width W3 is not used.
In the case of type 11, the section width W0 = POS0-0, W1 = POS1-POS0, W2 = POS2-POS1, and W3 = 19-POS2.

  The maximum value / minimum value detection unit 165 detects the maximum value and the minimum value of the pixel data in the sub block for each sub block. The maximum value / minimum value detection unit 165 outputs the detected maximum value and minimum value to the difference calculation unit 166 and the minimum value latch unit 167.

  The difference calculation unit 166 calculates the difference between the maximum value and the minimum value in the sub block detected by the maximum value / minimum value detection unit 165, and calculates the dynamic range (DR1) in units of sub blocks.

The minimum value latch unit 167 sequentially holds the minimum value in the register MINn (n = 0, 1, 2, 3) in units of sub blocks.
The selector 168 sequentially reads out and outputs the minimum value for each sub-block from the register MINn (n = 0, 1, 2, 3) of the minimum value latch unit 167.
The area binarized value holding unit 169 holds ARAn (n = 0, 1, 2, 3) = 0 or 1 output from the area binarizing unit 159.

  The quantization word length determination unit 170 determines the quantization word length in units of sub blocks. The quantization word length determination unit 170 outputs the quantization word length (Qn (n = 0, 1, 2, 3) = 2 to 5) for each sub-block. Here, the determined quantization word length is 2-5.

  According to the algorithm of the present embodiment, when a sub-block corresponds to a transition region to both ends of the maximum value and minimum value of the input waveform of one block and the section width of the sub-block is narrow, pixel data changes. Judged as a severe section. Then, it is assumed that a larger quantization word length is preferentially assigned to a sub-block that is determined to be a section in which the pixel data changes drastically.

  The quantized word length determination unit 170 determines the value of the section width Wn (n = 0, 1, 2, 3), AREAn (n = 0, 1, 2, 3) = 0 or 1, type (= 00, 01, 10, 11) is used to detect the order in which the quantized word length is assigned.

Of the word length assigned to one block (128 bits in this embodiment), the number of bits that can be used for quantized data differs for each type of compressed data having the data structure shown in FIG. is there.
type00: 100 bits type01: 93 bits type10: 74 bits type11: 55 bits

Based on the number of bits that can be used for these quantized data, a combination of quantized word lengths assigned to each pixel in one block can be determined as follows, for example.
type00: All 20 pixels are 5 bits (total of 100 bits)
type01: 5 bits up to 13 pixels, 4 bits for the remaining pixels (total 93 bits)
type10: 4 bits up to 14 pixels, 3 bits for the remaining pixels (74 bits in total)
type11: 3 bits up to 15 pixels, 2 bits for the remaining pixels (total 55 bits)

Furthermore, when the quantized word length assigned as in types 01, 10, and 11 differs depending on the pixel, the rule for assigning a larger quantized word length can be determined as follows.
1) Sub-blocks corresponding to AREAn = 0 (waveform transition region) and having a shorter section width W are assigned to the pixels having the larger quantization word length within the above-mentioned limit number.
2) Next, if there is still room to allocate the larger quantized word length, sub-blocks corresponding to ARAn = 1 and sub-blocks having a shorter section width W are within the above limit number. , The larger quantization word length is assigned to each pixel.
3) When the larger quantization word length is no longer assigned, the remaining sub-blocks are all assigned the smaller quantization word length.

  Next, an example of quantization word length assignment will be described with reference to FIG. FIG. 8 is an explanatory diagram showing an input waveform of pixel data. Here, a case will be described in which the block to be compressed is type 10 and division is performed so that the number of sub-blocks is 3.

  Since it is type10, according to the rule described above, the larger quantization word length is 4 bits, and a maximum of 14 pixels can be allocated. All the remaining pixels are assigned 3 bits.

According to the input waveform shown in FIG. 8, the section width W and the area AREA of each sub-block in the target block are as follows.
W0 = 7 AREA0 = 1
W1 = 3 AREA1 = 0
W2 = 10 AREA2 = 1

  At this time, the size relationship of the section width is W1 <W0 <W2. Therefore, in the above-described rule, the larger quantization word length is assigned in the order of W1, W0, and W2. First, Q0 = 4 bits are assigned to the section of W1. Next, a quantization word length is assigned to the interval of W0. Since 4 bits are allocated to a maximum of 14 pixels and W0 + W1 <14, Q1 = 4 bits are allocated to the section of W0. Next, a quantization word length is assigned to the section of W2. Although W2 = 10, since the quantum word length of 4 bits is already allocated for 10 pixels, 4 bits cannot be allocated to the section of W2. Therefore, Q2 = 3 bits are allocated to the section of W2. According to type 10, the section width W3 is not applicable because the number of sub-block divisions is 3, and Q3 is not calculated.

  Note that this quantization word length assignment procedure is also performed on the decompression processing side according to the same rule, and the inverse quantization processing is performed. In the present invention, the size and type of the quantization word length to be assigned are not limited to the above-described examples. At this time, the number of bits that can be used for the quantized data is limited depending on the type, but the same rule may be used on the compression processing side and the expansion processing side.

  The shift amount calculation unit 171 outputs the output of the subtraction unit 135 to the right based on the dynamic range (DR1) in units of sub-blocks and the quantization word length (Qn (n = 0, 1, 2, 3) = 2 to 5). Calculate how many bits to shift. The shift amount calculation unit 171 outputs the calculation result to the register SFTn (n = 0, 1, 2, 3) of the shift amount latch unit 172.

  The shift amount latch unit 172 sequentially records the shift amount output from the shift amount calculation unit 171 in the register SFTn (n = 0, 1, 2, 3). The selector 173 sequentially reads out and outputs the shift amount for each subblock from the register SFTn (n = 0, 1, 2, 3) of the shift amount latch unit 172.

Here, the shift amount calculation calculates the amount of right shift performed by the quantization unit 136 based on the dynamic range (DR1) of the region (subblock) to be compressed and the quantization word length to be output.
For example, when 10 bits are quantized to 5 bits,
When 0 ≦ DR1 ≦ 31, the input data (output data from the subtraction unit 135) is output as it is.
When 32 ≦ DR1 ≦ 63, the input data is shifted to the right by one bit.
When 64 ≦ DR1 ≦ 127, the input data is shifted to the right by 2 bits.
When 128 ≦ DR1 ≦ 255, the input data is shifted right by 3 bits.
When 256 ≦ DR1 ≦ 511, the input data is shifted right by 4 bits.
When 512 ≦ DR1 ≦ 1023, the input data is shifted right by 5 bits.

Also, when 10 bits are quantized to 4 bits, the process of “output as it is” to “shift to the right of 6 bits” is performed according to the dynamic range (DR1) as in the case of the 5 bits.
When quantizing 10 bits into 3 bits, processing of “output as it is” to “shift to the right of 7 bits” is performed according to the dynamic range (DR1). When quantizing 10 bits into 2 bits, processing of “output as it is” to “shift right by 8 bits” is performed according to the dynamic range (DR1).

Next, the RAW expansion unit 122 according to the present embodiment will be described with reference to FIG. FIG. 11 is a block diagram showing the RAW decompression unit 122 according to the present embodiment.
As illustrated in FIG. 11, the RAW decompression unit 122 includes, for example, a compression code latch unit 181, a section width calculation unit 182, a quantization word length determination unit 183, an address counter 184, a selector 185, a code shifter 186, a selector 187, and a selector. 188, an inverse quantization unit 189, an addition unit 191, and an inverse broken line extension unit 192.

  The compression code latch unit 181 holds 128-bit compressed data read from the SDRAM 104. The compression code latch unit 181 determines the type from 2 bits of MSB (Most Significant Bit) in the compressed data shown in FIG. 10, and holds the shift amount and the minimum value of each sub-block. The compression code latch unit 181 is an example of an extraction unit.

  The section width calculation unit 182 receives the coordinate data (POS0, 1, 2) of the sub-block switching coordinates, calculates the difference between the coordinate data (POS0, 1, 2), and calculates the section width W0, 1, 2. . The section width calculation unit 182 is the same as the section width calculation unit 164 of the RAW compression unit 121.

  The quantization word length determination unit 183 assigns and calculates the quantization word length of the subblock based on the section widths W0, 1, 2, 3 and the type calculated by the section width calculation unit 182 and calculates the quantum for each subblock. The chemical word length (Q0, 1, 2, 3) is determined. The quantized word length determining unit 183 determines the quantized word length (Q0, 1, 2, 3) for each sub-block using the same algorithm as the quantized word length determining unit 170 of the RAW compressing unit 121.

The selector 185 receives the quantized word length determined by the quantized word length determining unit 183 and sequentially outputs it to the code shifter 186.
The address counter 184 detects the switching coordinates of the sub-block to which the pixel to be expanded belongs based on the type and coordinate data, and sequentially switches the selectors 185, 187, and 188.

  The code shifter 186 receives quantized data (100 bits, 93 bits, 74 bits, 55 bits) corresponding to the type among the data held by the compression code latch unit 181. Then, the code shifter 186 extracts a code of 2 to 5 bits by a shift operation based on the quantization word length (Q0, 1, 2, 3) in units of sub-blocks, and outputs the code to the inverse quantization unit 189.

  The selector 187 receives the shift amount of each sub-block of the compressed data having the data structure of FIG. 10 held by the compression code latch unit 181, and sequentially outputs it to the inverse quantization unit 189. The selector 188 receives the minimum value of each sub-block of the compressed data having the data structure shown in FIG. 10 among the compressed data held by the compression code latch unit 181, and sequentially outputs it to the adder 191.

  The inverse quantization unit 189 inversely quantizes the quantized data for each pixel from the code shifter 186 according to the shift amount from the selector 187. In the present embodiment, the inverse quantization unit 189 inversely quantizes the 2 to 5 bit code and outputs 10 bit data. When the quantization unit 136 of the RAW compression unit 121 performs quantization as the right shifter, the inverse quantization unit 189 performs calculation as the left shifter as follows.

For example, when dequantizing 5 bits to 10 bits,
When 0 ≦ DR1 ≦ 31, the input data (output data from the code shifter 186) is output as it is.
When 32 ≦ DR1 ≦ 63, the input data is shifted to the left by one bit.
When 64 ≦ DR1 ≦ 127, the input data is shifted to the left by 2 bits.
When 128 ≦ DR1 ≦ 255, the input data is shifted left by 3 bits.
When 256 ≦ DR1 ≦ 511, the input data is shifted to the left by 4 bits.
When 512 ≦ DR1 ≦ 1023, the input data is shifted left by 5 bits.

Also, in the case of dequantizing 4 bits to 10 bits, as in the case of 5 bits, processing of “output as it is” to “shift to 6 bits left” is performed according to the dynamic range (DR1). .
When dequantizing 3 bits to 10 bits, processing of “output as it is” to “shift left by 7 bits” is performed according to the dynamic range (DR1). When 2 bits are inversely quantized to 10 bits, processing of “output as it is” to “shift left by 8 bits” is performed according to the dynamic range (DR1).

  The adder 191 adds the minimum value that is a common DC offset value for each sub-block to the code output from the inverse quantizer 189. The adding unit 191 outputs the data obtained by adding the minimum value to the reverse polygonal line extending unit 192.

  The reverse broken line decompression unit 192 decompresses the data from the addition unit 191 from 10 bits to 14 bits with the reverse characteristics to those of the broken line compression unit 131 of the RAW compression unit 121.

Here, FIG. 12 is a diagram illustrating an example of a polygonal line used in the reverse polygonal line extending unit 192.
The polygonal line in FIG. 12 is adapted to convert the gradation by characteristics opposite to those of the polygonal line compression unit 131 shown in FIG. A ROM table in which input data and decompressed output data are stored in association with each other may be prepared, and input / output data conversion may be performed according to the ROM table. If compression by the polygonal line compression unit 131 is not applied at the time of compression, data conversion at the reverse polygonal line conversion unit 192 is also bypassed at the time of decompression.

Next, the data structure of the compressed data generated in this embodiment will be described with reference to FIG. FIG. 10 is an explanatory diagram illustrating a data structure of compressed data for one block generated by the packing unit 138.
The compressed data is generated by the RAW compression unit of the present embodiment. 14 bits of the RAW data are converted into a 10-bit polygonal line compression unit, further divided into blocks of 20 pixels in the horizontal direction, and each pixel is quantized. It is encoded with a word length of 2 to 5 bits. The compressed data is composed of 128 bits for each block.

As shown in FIG. 10, compressed data of type 00 that is not divided into sub-blocks at the time of compression includes an identification flag (2 bits), a minimum value in the block (10 bits), and a dynamic range converted into, for example, a power of 2 in the block. (3 bits), consisting of block quantized data (100 bits).
On the other hand, in the case of compressed data of types 01, 10, and 11 divided into sub-blocks, the identification flag (2 bits), the minimum value in the sub-block for each sub-block (10 bits), for example 2 in the sub-block for each sub-block It consists of a dynamic range (3 bits) converted to a power of AREA, ARAn (1 bit) indicating a region, switching coordinates POSn (5 bits), and quantized data (93 bits, 74 bits, 55 bits).

  According to the RAW compression unit 121 and the RAW expansion unit 122 configured as described above, more appropriate sub-block division is selected so that image quality deterioration due to quantization is suppressed to a minimum, and quantization processing is executed. As a result, a high-quality captured image can be obtained by high-speed processing. Further, since the compression rate can be improved, the bandwidth of the internal bus 118 can be used effectively, and the response speed at the time of shooting by the continuous shooting function can be increased.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

1 is a block diagram illustrating an imaging apparatus according to a first embodiment of the present invention. It is a block diagram which shows the RAW compression part which concerns on the same embodiment. It is a block diagram which shows the block division number determination part which concerns on the same embodiment. It is a block diagram which shows the block division judgment part which concerns on the same embodiment. It is a block diagram which shows the data generation part for block division | segmentation which concerns on the same embodiment. It is a schematic diagram which shows a Bayer arrangement | sequence and 1 line of the same color extracted. It is explanatory drawing which shows the input waveform of pixel data. It is explanatory drawing which shows the input waveform of pixel data. It is a graph which shows the example of the broken line used in a broken line compression part. It is explanatory drawing which shows the data structure of the compressed data for 1 block produced | generated by the packing part. It is a block diagram which shows the RAW expansion | deployment part which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the broken line used in a reverse broken line extending | stretching part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image pick-up device 101 Image pick-up element 102 AFE circuit 103 Digital image processing part 104 SDRAM
105 ROM
106 Storage Device 111 Camera Signal Pre-Processing Unit 112 Camera Signal Processing Unit 113 Resolution Conversion Unit 114 JPEG Engine 115 CPU
116 video output encoder 117 SDRAM controller 121 RAW compression unit 122 RAW expansion unit

Claims (7)

A blocking unit that divides pixel data output from the image sensor into predetermined numbers and generates blocks;
A division processing unit that divides the block according to a level change of pixel data in the block and generates a sub-block;
A quantization unit that quantizes pixel data with a quantization word length determined according to a change amount of the pixel data level and a length of a section of the sub-block for each generated sub-block, and generates quantized data; ,
An image information processing apparatus, comprising: a compressed data generation unit that aggregates the quantized data and generates compressed data by combining information on a level change of pixel data in the block.   The image information processing apparatus according to claim 1, wherein the division processing unit divides the block and generates a sub-block according to an amplitude value and a change amount of a pixel data level in the block. The division processing unit
A division determination unit that determines the number and division position of the block based on the amplitude value and change amount of the pixel data level in the block;
The image information processing apparatus according to claim 1, further comprising: a sub-block generation unit configured to divide the block and generate a sub-block according to the determined division number and division position. From the compressed data, information on the level change of the pixel data in the block consisting of a predetermined number of pixel data, the amount of change in the pixel data level and the length of the section of the sub block for each sub block into which the block is divided An extraction unit for extracting quantized data obtained by quantizing the pixel data with a quantized word length determined according to the method;
An image information processing apparatus comprising: an inverse quantization unit that inversely quantizes the quantized data and generates inversely quantized pixel data based on information on the level change.   The image information processing apparatus according to claim 4, wherein the information related to the level change is information related to an amplitude value and a change amount of the level of the pixel data. Dividing the pixel data output from the image sensor every predetermined number to generate a block;
Dividing the block according to a level change of pixel data in the block to generate a sub-block;
Quantizing pixel data with a quantization word length determined according to a change amount of the pixel data level and a length of a section of the sub-block for each generated sub-block, and generating quantized data;
An image information processing method comprising: information on a level change of pixel data in the block; and a step of collecting the quantized data and generating compressed data. From the compressed data, information on the level change of the pixel data in the block consisting of a predetermined number of pixel data, the amount of change in the pixel data level and the length of the section of the sub block for each sub block into which the block is divided Extracting the quantized data obtained by quantizing the pixel data with the quantized word length determined accordingly;
An image information processing method comprising: dequantizing the quantized data and generating dequantized pixel data based on the information on the level change.

Images