JP2010074597A - Image processing device, image processing method, program and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device capable of: random access to the data corresponding to an arbitrary area in image data; and improving the compression efficiency of the image data, an image processing method, a program and an imaging device. <P>SOLUTION: A compression section included in the image processing device includes: a frequency measuring portion for measuring the frequency of the dynamic range of the block consisting of pixel data of a predetermined number with regard to image data; a quantization word length allocation calculation portion for computing the quantization word length assigned to each dynamic range for every block based on the frequency; and a padding portion for adding 0 or 1 until reaching a predetermined code amount with respect to the compressed image data including the coding data for every block quantized based on the computed quantization word length by predetermined blocks. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing device, an image processing method, a program, and an imaging device.

  2. Description of the Related Art In recent years, imaging apparatuses capable of capturing an image using a solid-state image sensor, such as a digital still camera and a digital video camera, and storing the obtained captured image as digital data are widely used. In such an image pickup apparatus, the number of pixels of the image sensor is increased, and the function and performance of the apparatus are increased. In particular, when the number of pixels of the image sensor is increased, the processing load of the image signal increases. On the other hand, in order to prevent the user from feeling stress regarding operability, it is necessary to increase the processing speed.

  Here, in a general imaging device, RAW data output from an imaging device is temporarily stored in an image memory such as SDRAM (Synchronous Dynamic Random Access Memory), and then read and subjected to camera signal processing and the like. . Therefore, in Patent Document 1 shown below, a time required for reading and writing image data in an image memory is shortened to realize a high-speed imaging operation.

JP 2007-228515 A

  However, in the method described in Patent Document 1, since fixed length compression is performed on image data, random access to data corresponding to an arbitrary area in the image data is possible. There has been a problem that it is difficult to further increase the compression efficiency.

  Therefore, the present invention has been made in view of such a problem, and an object of the present invention is to enable random access to data corresponding to an arbitrary area in the image data and to improve the compression efficiency of the image data. It is an object of the present invention to provide a new and improved image processing apparatus, image processing method, program, and imaging apparatus that can be improved.

  In order to solve the above-described problem, according to an aspect of the present invention, the image processing apparatus includes a compression unit that compresses image data output from an image sensor, and a storage unit that holds the compressed image data. The compression unit, for the image data, a frequency measurement unit that measures the frequency of a dynamic range of a block made up of a predetermined number of pixel data, and a quantized word assigned to each dynamic range for each block based on the frequency A quantized word length allocation calculating unit for calculating a length, and a predetermined code for compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length An image processing apparatus is provided that includes a padding unit that adds 0 or 1 until the amount reaches.

  According to such a configuration, the frequency measurement unit included in the compression unit of the image processing apparatus measures the frequency of the dynamic range of the block including a predetermined number of pixel data for the image data. Further, the quantized word length allocation calculation unit calculates the quantized word length allocated to each dynamic range for each block based on the measured frequency. The padding unit adds 0 or 1 to the compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantization word length until the predetermined code amount is reached. To do.

  The image processing apparatus holds in advance a quantization word length table in which a plurality of patterns having different combinations of quantization word lengths assigned to each dynamic range are described, and the quantization word length assignment calculation unit includes: , Calculating a determined code amount of compressed image data for the predetermined block in each pattern based on the plurality of patterns described in the quantized word length table and the frequency, and the determined code amount is a desired code amount It is preferable to select a combination of the quantized word lengths in the pattern to be as follows.

  The quantized word length allocation calculating unit may set the quantized word length allocated to each dynamic range to a predetermined fixed value when the determined code amount exceeds the desired code amount.

  The quantized word length allocation calculating unit may transmit a difference between the predetermined code amount and the determined code amount in the selected combination of quantized word lengths as a padding amount to the padding unit.

  The compression unit further includes a quantization execution unit that quantizes the image data for each block based on the calculated quantization word length, and the quantization execution unit has a predetermined number of bits, Generating encoded data composed of a fixed length part including at least information indicating the calculated quantization word length and a variable length part including the encoded data corresponding to the predetermined number of pixel data; Also good.

  The image processing apparatus further includes a decompression unit that obtains the compressed image data held in the storage unit and decompresses data corresponding to an arbitrary rectangular area from the compressed image data, The decompression unit may specify the position of data corresponding to the arbitrary rectangular region based on information representing the quantization word length included in the fixed length unit.

  In order to solve the above problems, according to another aspect of the present invention, a step of measuring a frequency of a dynamic range of a block composed of a predetermined number of pixel data for image data output from an image sensor, And calculating a quantized word length allocated to each dynamic range for each block, and encoding the encoded data for each block quantized based on the calculated quantized word length for a predetermined block. And adding 0 or 1 to the compressed image data including a predetermined code amount.

  In order to solve the above problems, according to still another aspect of the present invention, image data output from an image sensor is input, and the input image data is input to a computer capable of compressing and storing the input image data. For the image data, a frequency measurement function for measuring the frequency of a dynamic range of a block composed of a predetermined number of pixel data, and a quantization word length assigned to each dynamic range is calculated for each block based on the frequency. Quantized word length allocation calculation function and compressed image data including predetermined blocks of encoded data for each block quantized based on the calculated bona fide quantized word length until a predetermined code amount is reached A program for realizing a padding function for adding 0 or 1 is provided.

  According to this configuration, the computer program is stored in the storage unit included in the computer, and is read and executed by the CPU included in the computer, thereby causing the computer to function as the image processing apparatus. A computer-readable recording medium in which a computer program is recorded can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

  In order to solve the above-described problem, according to still another aspect of the present invention, a compression unit that compresses image data output from an image sensor and a storage unit that stores the compressed image data are provided. The compression unit includes a frequency measurement unit that measures the frequency of a dynamic range of a block including a predetermined number of pixel data for the image data, and a quantum allocated to each dynamic range for each block based on the frequency. A quantized word length allocation calculating unit that calculates a quantized word length, and a predetermined amount of compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length An imaging device is provided that includes a padding unit that adds 0 or 1 until the amount of codes becomes.

  According to the present invention, random access to data corresponding to an arbitrary area in image data is possible, and the compression efficiency of image data can be improved.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
(1) About conventional imaging device (2) First embodiment About configuration of imaging device
About the overall configuration of the imaging device
About the configuration of the RAW compression unit
Configuration of RAW decompression unit (3) Summary Summary on compression processing Summary on decompression processing

<Regarding conventional imaging apparatus>
First, a conventional imaging device will be described in detail with reference to FIG. FIG. 14 is a block diagram for explaining the configuration of a conventional imaging apparatus.

  A conventional imaging apparatus 900 is, for example, a digital still camera or a digital video camera, and can record a subject image as digital data. In the following description, the case where the imaging apparatus 900 is a digital still camera will be described as an example.

  As illustrated in FIG. 14, the imaging apparatus 900 includes, for example, an imaging element 901, an analog front end (AFE) circuit unit 903, a digital image processing circuit unit 905, a ROM (Read Only Memory) 919, and a storage device 921. And SDRAM 929 and the like.

  The digital image processing circuit unit 905 includes a camera signal preprocessing unit 907, a camera signal processing unit 911, a resolution conversion unit 913, a JPEG engine 915, a CPU (Central Processing Unit) 917, a video output encoder 923, an SDRAM controller 927, and the like. Have. Each of these components is connected to each other by an internal bus 909.

  The imaging device 901 is a solid-state imaging device such as a CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor), and converts light incident from a subject through an unillustrated lens block (optical system) into an electrical signal.

  The AFE circuit unit 903 performs sample hold on the image signal output from the image sensor 901 so as to maintain a good S / N (Signal / Noise) ratio by correlated double sampling (CDS) processing. Do. The AFE circuit 903 controls the gain (gain) by automatic gain control (Automatic Gain Control: AGC) processing, performs A / D (Analog / Digital) conversion, and outputs a digital image signal.

  The digital image processing circuit unit 905 is formed as, for example, a SoC (System on a Chip) circuit. In the digital image processing circuit unit 905, the camera signal preprocessing unit 907 performs signal correction processing for defective pixels in the image sensor 901 and shading for correcting the peripheral light loss of the lens with respect to the image signal supplied from the AFE circuit unit 903. Apply processing. The camera signal preprocessing unit 907 outputs the processed signal as RAW data.

  The camera signal processing unit 911 includes signal detection processing for automatic focus (Auto Focus: AF) control, automatic exposure (Auto Exposure: AE) control, white balance control, and signal correction processing represented by white balance adjustment. , So-called camera signal processing, or a part of the processing is executed. Further, the camera signal processing unit 911 converts the image data after the signal correction into a luminance signal (Y) and a color difference signal (RY, BY) in a predetermined format such as 4: 2: 2.

  The resolution conversion unit 913 receives image data processed by the camera signal processing unit 911 or image data decompressed and decoded by a JPEG engine 915 described later, and converts the image data to a predetermined resolution.

  The JPEG engine 915 compresses and encodes the image data processed by the resolution conversion unit 913, and generates JPEG encoded data. The JPEG engine 915 decompresses and decodes JPEG image data read from the storage device 921. Note that the digital image processing circuit unit 905 may be provided with an encoding / decoding engine of a still image compression method or a moving image compression method other than the JPEG engine 915.

  The CPU 917 performs overall control of the digital image processing circuit unit 905 and the entire imaging apparatus 900 by executing a program stored in a ROM 919 described later, and executes various calculations for the control.

  The ROM 919 holds programs executed by the CPU 917 and various data. The ROM 919 is a non-volatile memory such as an EEPROM (Electronically Erasable and Programmable ROM) or a flash memory, for example.

  The storage device 921 is a device for recording a file of encoded image data. The storage device 921 includes, for example, a recording medium such as a flash memory, various optical disks, and a magnetic tape, and a recording / reproducing drive for the recording medium.

  The video output encoder 923 is, for example, an NTSC (National Television Standards Committee) encoder. The video output encoder 923 generates an image signal for monitor display based on the image data output from the resolution conversion unit 913 and the like, and outputs the image signal to a monitor (not shown) or the video output terminal 925.

  The SDRAM controller 927 is an interface block for an SDRAM 929, which will be described later, and includes an address decoder and the like. The SDRAM controller 927 controls writing and reading operations of the SDRAM 929 according to a control signal from the CPU 917.

  The SDRAM 929 is a volatile memory such as a DDR (Double Data Rate) SDRAM. The SDRAM 929 is a work area for data processing in the digital image processing circuit unit 905. The SDRAM 929 includes a capture data area 931, a JPEG code area 933, a CPU work area 935, and the like. The capture data area 931 temporarily stores RAW data that is data captured from the image sensor 901. The JPEG code area 933 temporarily stores image data encoded by the JPEG engine 915, data used in the encoding / decoding process, and the like. The CPU work area 935 temporarily stores data used in the processing of the CPU 917.

  In the imaging apparatus 900 as described above, an imaging signal from the imaging element 901 is sequentially transmitted to the AFE circuit unit 903, subjected to CDS processing and AGC processing, converted into a digital signal, and then converted into a digital image processing circuit unit 905. To the camera signal pre-processing unit 907. The camera signal preprocessing unit 907 generates RAW data obtained by performing defective pixel correction, shading correction, or the like on the input image signal. This RAW data is temporarily written in the SDRAM 929.

  When the RAW data is read from the SDRAM 929, various image quality correction processes are performed by the camera signal processing unit 911. The processed image data is temporarily stored in the SDRAM 929, for example, and then converted into data having a resolution suitable for display by the resolution conversion unit 913. Further, the image data after resolution conversion is stored in, for example, the SDRAM 929 and then transmitted to the video output encoder 923. Thereby, a camera through image is displayed on the monitor.

  When image recording is requested through an input unit (not shown), the resolution conversion unit 913 converts the image data processed by the camera signal processing unit 911 into data having a resolution set for recording as necessary. For example, the data is converted and temporarily stored in the SDRAM 929. The JPEG engine 915 compresses and encodes image data to generate encoded data. For example, the encoded data is once recorded in the SDRAM 929 and then recorded in the storage device 921.

  Further, the image data (encoded data) recorded in the storage device 921 is decompressed and decoded by the JPEG engine 915 and the resolution is converted by the resolution converter 913. Thereafter, the image data is output to the video output encoder 923 so that the image can be displayed on the monitor.

  Here, when a RAW data write process or a read process is performed between the digital image processing circuit unit 905 and the SDRAM 929, capture size RAW data is transferred via the internal bus 909 in the digital image processing circuit unit 905. Will be. Therefore, if RAW data can be compressed and stored in the memory, more RAW data can be captured. In addition, since the RAW data is captured and read out, data for the entire screen flows on the internal bus 909, and therefore occupies a considerable portion of the bus bandwidth at the time of shooting. Therefore, if the data size is reduced due to the compression of RAW data, the bus bandwidth used at the time of shooting can be reduced.

  In addition, in the so-called continuous shooting function or the like, the upper limit of the number of continuous shots can be increased by following a procedure in which a plurality of frames of captured images are pasted on a memory at high speed and then signal processing is performed sequentially. Further, depending on the continuous shooting interval and the signal processing speed, it is possible to perform infinite continuous shooting even though the storage capacity is up to the upper limit.

  Thus, reduction of the transfer bus bandwidth of the RAW signal is a factor that greatly affects the performance of the imaging apparatus such as a digital still camera or a digital video camera.

  Therefore, in the above-mentioned Patent Document 1, when RAW data is compressed, fixed length compression of RAW data is performed, the bus bandwidth is fixed, and random access for each block can be realized. It couldn't be high.

  Further, although it is conceivable to compress the RAW data by lossless compression, if an element having a variable length such as a Huffman code is incorporated, the code amount is not fixed until the compression of one screen is completed. For this reason, there is a problem that the capture memory size cannot be reduced and a failure process such as overflow of code amount is required after capturing one frame. In addition, when it is desired to expand an arbitrary part of the compressed screen, data cannot be accessed unless all frames are decoded and then returned in the raster order. Therefore, there is a problem that processing time and a memory area are required.

  Therefore, the inventor of the present application has intensively studied in order to solve the above-described problems, and has come up with an image processing apparatus and an image processing method as described below.

(First embodiment)
<About the configuration of the imaging device>
Next, the configuration of an imaging apparatus that is an example of an image processing apparatus according to the first embodiment of the present invention will be described in detail with reference to FIGS.

[Overall configuration of imaging device]
First, the overall configuration of the imaging apparatus according to the present embodiment will be described in detail with reference to FIG. FIG. 1 is a block diagram for explaining a configuration of an imaging apparatus which is an example of an image processing apparatus according to the present embodiment.

  The imaging apparatus 10 according to the present embodiment is a digital still camera, a digital video camera, or the like, for example, and can record a subject image as digital data. In the following description, the case where the imaging device 10 is a digital still camera will be described as an example.

  As shown in FIG. 1, the imaging apparatus 10 includes, for example, an imaging device 11, an AFE circuit unit 13, a digital image processing circuit unit 15, a ROM 29, a storage device 31, an SDRAM 39, and the like.

  The digital image processing circuit unit 15 includes a camera signal preprocessing unit 17, a camera signal processing unit 21, a resolution conversion unit 23, a JPEG engine 25, a CPU 27, a video output encoder 33, an SDRAM controller 37, and the like. Furthermore, the digital image processing circuit unit 15 according to the present embodiment includes a RAW compression unit 101 and a RAW expansion unit 103. Each of these components is connected to each other by an internal bus 19.

  The imaging device 11 is a solid-state imaging device such as a CCD or CMOS, and converts light incident from a subject through an unillustrated lens block (optical system) into an electrical signal.

  The AFE circuit unit 13 samples and holds the image signal output from the image sensor 11 so as to maintain a good S / N ratio by CDS processing. The AFE circuit 13 controls the gain by AGC processing, performs A / D conversion, and outputs a digital image signal.

  The digital image processing circuit unit 15 is formed as a SoC circuit, for example. In the digital image processing circuit unit 15, the camera signal preprocessing unit 17 performs a signal correction process for a defective pixel in the image sensor 11 and a shading that corrects a decrease in the peripheral light amount of the lens for the image signal supplied from the AFE circuit unit 13. Apply processing. In addition, the camera signal preprocessing unit 17 outputs the processed signal as RAW data.

  The RAW compression unit 101 compresses the RAW data transmitted from the camera signal preprocessing unit 17 by a compression method described later, and transmits the compressed data to the SDRAM 39 via the SDRAM controller 37.

  The RAW decompression unit 103 decompresses the compressed RAW data read from the SDRAM 39 via the SDRAM controller 37 by a method described later and outputs the decompressed data to the camera signal processing unit 21.

  The camera signal processing unit 21 executes so-called camera signal processing such as signal detection processing for AF control, AE control, white balance control, signal correction processing represented by white balance adjustment, or a part of the processing. To do. The camera signal processing unit 21 converts the image data after signal correction into a luminance signal (Y) and color difference signals (RY, BY) in a predetermined format such as 4: 2: 2.

  The resolution conversion unit 23 receives image data processed by the camera signal processing unit 21 or image data decompressed and decoded by the JPEG engine 25 and converts the image data into a predetermined resolution.

  The JPEG engine 25 compresses and encodes the image data processed by the resolution converter 23 to generate JPEG encoded data. The JPEG engine 25 decompresses and decodes JPEG image data read from the storage device 31. The digital image processing circuit unit 15 may be provided with an encoding / decoding engine of a still image compression method or a moving image compression method other than the JPEG engine 25.

  The CPU 27 comprehensively controls the digital image processing circuit unit 15 and the entire imaging apparatus 10 by executing a program stored in a ROM 29 described later, and executes various calculations for the control.

  The ROM 29 holds programs executed by the CPU 27 and various data. The ROM 29 is a nonvolatile memory such as an EEPROM or a flash memory, for example.

  The storage device 31 is a device for recording an encoded image data file. The storage device 31 includes, for example, a recording medium such as a flash memory, various optical disks, and a magnetic tape, and a recording / reproducing drive for the recording medium.

  The video output encoder 33 is, for example, an NTSC encoder. The video output encoder 33 generates an image signal for monitor display based on the image data output from the resolution conversion unit 23 and the like, and outputs it to a monitor (not shown) or the video output terminal 35.

  The SDRAM controller 37 is an interface block for an SDRAM 39 to be described later, and includes an address decoder and the like. The SDRAM controller 37 controls writing and reading operations of the SDRAM 39 according to a control signal from the CPU 27.

  The SDRAM 39 is a volatile memory such as a DDR SDRAM, and is an example of a storage unit included in the imaging device 10. The SDRAM 39 is a work area for data processing in the digital image processing circuit unit 15. The SDRAM 39 includes a capture data area 41, a JPEG code area 43, a CPU work area 45, and the like. The capture data area 41 temporarily stores RAW data that is data captured from the image sensor 11. The JPEG code area 43 temporarily stores image data encoded by the JPEG engine 25, data used in the encoding / decoding process, and the like. The CPU work area 45 temporarily stores data used for processing by the CPU 27.

  In the image pickup apparatus 10 as described above, the image pickup signal from the image pickup element 11 is sequentially supplied to the AFE circuit unit 13, subjected to CDS processing and AGC processing, converted into a digital signal, and then converted into a digital image processing circuit unit 15. To the camera signal pre-processing unit 17. The camera signal preprocessing unit 17 generates RAW data obtained by performing defective pixel correction, shading correction, or the like on the input image signal. This RAW data is compressed by the RAW compression unit 101 and then temporarily written in the SDRAM 39.

  When the RAW data is read from the SDRAM 39, it is expanded by the RAW expansion unit 103 and then subjected to various image quality correction processes by the camera signal processing unit 21. The processed image data is temporarily stored in, for example, the SDRAM 39 and then converted into data having a resolution suitable for display by the resolution conversion unit 23. The image data after resolution conversion is supplied to the video output encoder 33 after being stored in the SDRAM 39, for example. Thereby, a camera through image is displayed on the monitor.

  When image recording is requested through an input unit (not shown) or the like, the resolution conversion unit 23 converts the image data processed by the camera signal processing unit 21 into data having a resolution set for recording as necessary. For example, it is temporarily stored in the SDRAM 39 after conversion. The JPEG engine 25 compresses and encodes image data to generate encoded data (hereinafter also referred to as compressed code data). For example, the encoded data is once recorded in the SDRAM 39 and then recorded in the storage device 31.

  The image data (compressed code data) recorded in the storage device 31 is decompressed and decoded by the JPEG engine 25 and the resolution is converted by the resolution converter 23. Thereafter, the image data is output to the video output encoder 33 so that the image can be displayed on the monitor.

  The digital image processing circuit unit 15 of the present embodiment is provided with a RAW compression unit 101 that compresses RAW data at an input position of image data from the camera signal preprocessing unit 17 to the internal bus 19. Thereby, the amount of RAW data transmitted to the SDRAM 39 through the internal bus 19 can be reduced. In addition, a RAW expansion unit 103 that expands RAW data is provided at an input position of image data from the internal bus 19 to the camera signal processing unit 21. As a result, the amount of raw data transmitted from the SDRAM 39 to the camera signal processing unit 21 can be reduced.

  According to the present embodiment, it is possible to reduce the transmission load of the internal bus 19 during the imaging operation, and to shorten the time required for the write / read processing for the SDRAM 39. In particular, by simplifying the compression / decompression process as much as possible, the effect of shortening the processing time can be increased. In addition, power consumption can be suppressed by reducing the transmission frequency on the bus.

  Further, according to the present embodiment, the capacity of the SDRAM 39 can be reduced. In addition, use of the SDRAM 39 area for other processing, storing RAW data for multiple frames, increasing the number of continuous shots, or increasing the continuous shooting speed, etc. Contribute to. Therefore, it is possible to realize a high-performance, small, and low-cost imaging device 10 that can shorten the time required for the imaging process and the data recording process.

  The installation positions of the RAW compression unit 101 and the RAW expansion unit 103 shown in FIG. 1 are merely examples, and may be installed on the SDRAM controller 37 side depending on the system configuration.

[Configuration of Raw Compression Unit]
Next, the configuration of the RAW compression unit 101, which is an example of the compression unit according to the present invention, will be described in detail with reference to FIGS. The RAW compression unit 101 according to the present embodiment is a compression unit using an irreversible RAW signal compression method.

  In the following description, it is assumed that the RAW data signal is 14 bits per pixel. Further, for example, as shown in FIG. 5, 16 pixels in the horizontal direction having the same color component are converted into one block of compressed code data. FIG. 5 is an explanatory diagram for explaining the Bayer array and the extracted one line of the same color. FIG. 5 will be described in detail later. The RAW compression unit 101 according to the present embodiment compresses a 15-bit RAW data signal to 10 bits by a polygonal line compression unit 121, which will be described later, and then performs quantization in block units with a quantization word length Q = 1 to 6 bits. Do. Here, the quantized word length Q is the same within one block, and the quantized word length Q varies from block to block. In the following, a case where the target code amount is set to 6 bits (6 bits / pix) per pixel will be described.

  The RAW compression unit 101 according to the present embodiment acquires the difference between the maximum value and the minimum value of the pixel data in units of blocks, and 32 to 112 bits (32 to 112 bits / BLOCK per block) according to the size of the dynamic range. ) To a variable length. Also, the RAW compression unit 101 according to the present embodiment always compresses to 6 bits per pixel (6 bits / pix) by a quantized word length allocation calculation unit 127 described later. Therefore, the RAW compression unit 101 according to the present embodiment can realize compression with a fixed length of 6 bits / pix for each frame.

  FIG. 2 is a block diagram for explaining the configuration of the RAW compression unit according to this embodiment. As shown in FIG. 2, for example, the RAW compression unit 101 according to the present embodiment includes a polygonal line compression unit 121, a frequency measurement unit 123, a one-line delay unit 125, a quantization word length allocation calculation unit 127, Mainly includes a conversion execution unit 129 and a padding unit 131.

  The polygonal line compression unit 121 nonlinearly compresses the input 14-bit RAW data into 10-bit data by approximation using a polygonal line. The polygonal line compression unit 121 is provided for the purpose of improving the overall compression efficiency by lowering the gradation as much as possible before the subsequent compression procedure. Depending on the target compression rate, it may be omitted. In this case, the reverse broken line extension unit 231 provided at the output stage of the RAW extension unit 103 described later with reference to FIG. 12 can also be omitted.

  Here, FIG. 3 is a graph showing an example of a broken line used in the broken line compression unit 121. The polygonal line compression unit 121 converts the gradation of the input data using the graph shown in FIG. In this example, in accordance with human visual characteristics, a higher gradation is assigned as input data is smaller, that is, darker (or lighter in color). Such a polygonal line may be prepared for each color component, for example, and used by switching for each color component of the input pixel.

  In the polygonal line compression unit 121, for example, after converting the gradation of the input data using such a polygonal line, the converted data is divided by 16 (that is, shifted down by 4 bits) to become 10-bit data. Compress. At this time, the discarded lower bits are rounded off, for example. A ROM table that stores input data and compressed output data in association with each other based on the above calculation is prepared, and the polygonal line compression unit 121 converts input / output data according to the ROM table. Also good. The broken line compression unit 121 may be an arithmetic circuit that sets, for example, about four broken points.

  The RAW data compressed by the polygonal line compression unit 121 is transmitted to the frequency measurement unit 123 described later. The timing at which the compressed code data is transmitted to the frequency measuring unit 123 is transmitted to the 1-line delay unit 125 described later.

  The frequency measurement unit 123 detects the maximum value and the minimum value of the pixel data for each block unit, and calculates the dynamic range from the difference between the detected maximum value and the minimum value. Subsequently, the frequency measurement unit 123 detects the calculated word length (number of digits) of the dynamic range, and measures the frequency of the word length DR of the dynamic range. The frequency measurement period by the frequency measurement unit 123 is, for example, a period corresponding to one line. In the present embodiment, there are ten dynamic range word lengths DR from DR = 1 to DR = 10. The frequency measuring unit 123 transmits the frequency of the word length DR of each dynamic range to the quantized word length allocation calculating unit 127 as histograms HIST1 to HIST10.

  Here, FIG. 4 is a block diagram for explaining a detailed configuration of the frequency measurement unit 123. For example, as shown in FIG. 4, the frequency measurement unit 123 according to the present embodiment includes a blocking unit 141, a maximum / minimum detection unit 143, a dynamic range calculation unit 145, a word length detection unit 147, and a comparator 149A. To 149J and counters 151A to 151J.

  The blocking unit 141 divides the data transmitted from the polygonal line compression unit 121 into blocks composed of the same color component pixels for 16 pixels adjacent in the horizontal direction, and outputs the divided blocks. Thereby, the correlation of the data in a block becomes strong, and the image quality deterioration by subsequent quantization processing can be suppressed.

For example, when a Bayer array image sensor as shown in FIG. 5 is used, in the output data, the repetition of the R component and the Gr component and the repetition of the B component and the Gb component appear for each line. For example, when the R component and the Gr component repeatedly appear in the input data to the blocking unit 141 as R ₀ , Gr ₀ , R ₁ , Gr ₁ ,..., R ₁₅ , Gr ₁₅ , _{0, R 1, R 2,} ···, R 15, Gr 0, Gr 1, ···, as _{Gr 15,} so that the pixel of the same color component appears 16 continuously converts the output order To block.

  The maximum / minimum detection unit 143 detects the maximum value (MAX) and the minimum value (min) from the block data of 16 horizontal pixels for each block. The maximum / minimum detection unit 143 transmits the detected maximum value (MAX) and minimum value (min) to the dynamic range calculation unit 145.

  The dynamic range calculation unit 145 calculates the difference (MAX−min) between the maximum value (MAX) and the minimum value (min) in one block, and calculates the dynamic range in one block. The dynamic range calculation unit 145 transmits the calculated dynamic range to the word length detection unit 147.

  The word length detection unit 147 detects the number of digits of the dynamic range of the pixel data existing in one block (that is, the word length of the dynamic range) DR from the transmitted dynamic range. The word length detection unit 147 outputs the detected number DR of the dynamic range to the comparators 149A to 149J described later.

  The comparators 149A to 149J compare the dynamic range digit number DR output from the word length detection unit 147 with the reference value input to each comparator. Each comparator outputs 1 to the counter connected to each comparator when the dynamic range is equal to the input reference value. In this embodiment, since there are ten dynamic range digits DR, DR = 1 to DR = 10, 1 to 10 are input to the comparators 149A to 149J as reference values, respectively. Since the number of digits DR of the dynamic range transmitted from the word length detection unit 147 has any value of 1 to 10, any one of the comparators 149A to 149J outputs a signal representing 1.

  For example, when the number of digits DR of the dynamic range output from the word length detection unit 147 is 1, the output of the comparator 149A to which the reference value 1 is input is 1, and the outputs of the comparators 149B to 149J are 0. .

  The counters 151A to 151J count the number of times a signal representing 1 is transmitted from the corresponding comparators 149A to 149J. The number of times output from these counters 151A to 151J is the frequency of the dynamic range digit number DR, that is, the histograms HIST1 to HIST10. The values output from the counters 151A to 151J are transmitted to the quantized word length allocation calculating unit 127.

  For example, the number of times output from the counter 151A is equal to the number of times that a signal representing 1 is output from the comparator 149A. That is, the value output from the counter 151A represents the number of blocks whose dynamic range digit number DR is 1.

  Using the method described above, the frequency measurement unit 123 measures the frequency of the dynamic range digit number DR.

  Returning to FIG. 2 again, the configuration of the RAW compression unit 101 according to the present embodiment will be described in detail.

  The one-line delay unit 125 is a delay line having a length of one horizontal synchronization period (1H). A period of one line is required after the frequency measurement unit 123 starts measuring DR of the input data until the quantized word length allocation calculating unit 127 determines the quantized word length. Therefore, the 1-line delay unit 125 is used to match the time until the output from the polygonal line compression unit 121 can be quantized.

  Based on the frequency transmitted from the frequency measurement unit 123, the quantization word length allocation calculation unit 127 calculates the quantization word length allocated to each of the dynamic range digits DR = 1 to DR = 10 for each block. . More specifically, the quantized word length allocation calculating unit 127 compresses a predetermined block of compressed images in each pattern based on a plurality of patterns described in a quantized word length table to be described later and the transmitted frequency. Calculate the definite code amount of the data. Subsequently, the quantized word length allocation calculating unit 127 selects a quantized word length combination in a pattern in which the calculated determined code amount is equal to or smaller than a desired code amount as a quantized word length used for data compression.

  The quantized word length table describes a plurality of patterns having different combinations of quantized word lengths assigned to the number of digits DR = 1 to DR = 10 of each dynamic range. The quantized word length table is stored in advance in the ROM 29 according to the present embodiment, for example.

  FIG. 6 shows an example of the quantization word length table as described above. The table shown in FIG. 6 describes 13 patterns in which the combinations of quantization word lengths assigned to the number of digits DR of each dynamic range are set to change according to the number of digits. In addition to the above 13 patterns, the table shown in FIG. 6 also describes a pattern (corresponding to the index 13) in which the same quantization word length is assigned to the number of digits DR of all dynamic ranges. .

  For example, index 0 to index 3 in FIG. 6 are patterns intended to assign quantization word lengths with as little code amount as possible to those having a small number of dynamic range digits DR. In addition, the index 4 to the index 12 in FIG. 6 are patterns intended to suppress the generated code amount of a block with a small number of digits and to allocate the suppressed amount to a block having a dynamic range with a large number of digits. . As is clear from FIG. 6, the smaller the index value, the closer the quantized word length is to the value of the dynamic range digit number DR. This means that the smaller the index value, the better the image quality after compression.

  As is clear from FIG. 6, in the patterns described in the index 0 to index 12, the values of the same quantized word length are not obtained for all the dynamic ranges. Therefore, it is understood that variable length compression is performed in units of blocks by using a combination of these quantized word lengths. Also, in the pattern described in the index 13, the same quantized word length value is used for all the dynamic range digits. Therefore, it can be seen that by using this combination of quantized word lengths, fixed length compression is performed in block units (and in line units).

  Here, in the RAW compression unit 101 according to the present embodiment, for example, as shown in FIG. 8, the compression code data format of one block is converted into a quantized data format consisting of two types of parts, a fixed part and a variable length part. To do. In this data format, the number of pixels in one block and the word length alignment of the fixed part are equal. For example, when one block is composed of 16 pixels as in this embodiment, the word length of the fixed part is 16 bits or a multiple thereof, and when the quantized word length is Q bits, the quantized data that becomes the variable length part The part is set to 16 × Q bits.

  By using such a compressed code data format, the quantized word length allocation calculation unit 127 performs the calculation shown in the following Equation 1 and calculates the determined value of the code amount without performing compression processing. Can do.

Deterministic code amount of the number of digits DR of the dynamic range =
Sum of 1 to 10 (frequency for each DR × quantized word length for each DR) + (number of blocks in one line × number of words in a fixed part)
... (Formula 1)

  Here, in the above equation 1, the quantization word length for each DR is a value described in the quantization word length table as shown in FIG. 6, and the number of blocks in one line and the number of words in the fixed part are This is a value set in advance in the imaging apparatus. Further, in Equation 1 above, the frequency for each DR is a value transmitted from the frequency measurement unit 123.

  The quantized word length allocation calculating unit 127 calculates the deterministic code amount for all patterns other than those that are fixed length compression described in the quantized word length table, using Equation 1 above. On the other hand, the quantized word length allocation calculating unit 127 calculates the required code amount as shown in the following Expression 2, using the set target code amount.

  Required code amount = (target code amount × total number of pixels in one line) (Expression 2)

  For example, when the target code amount is 6 bits / pix as in the present embodiment, the required code amount is 6 bits × the total number of pixels of one line from Equation 2.

  Subsequently, the quantized word length allocation calculation unit 127 compares the calculated determined code amount with the requested code amount. As a result of the comparison, the quantized word length allocation calculation unit 127 selects a pattern (index) that is equal to or smaller than the required code amount and has the largest deterministic code amount as a combination of quantized word lengths to be selected. Further, as shown in FIG. 6, if the table is arranged so that the smaller the index value is, the better the image quality is, and the table has a determined code amount equal to or less than the required code amount and the smallest index value. You may make it select.

  When the selection of the quantization word length combination pattern is completed, the quantization word length allocation calculating unit 127 selects the combination of the selected quantization word lengths (that is, the quantization words described in the selected quantization word length table). The combination of the long Q1 to Q10) is transmitted to the quantization execution unit 129. For example, when a quantized word length table having an index value of 3 is selected, the quantized word length allocation calculating unit 127 performs Q1 = 1, Q2 = 2, Q3 = 3, Q4 = 4, Q5 = 5, Q6. = 5, Q7 = 5, Q8 = 5, Q9 = 6, and Q10 = 6 are transmitted to the quantization execution unit 129.

  If the calculated determined code amounts are all larger than the requested code amount, the quantized word length assignment calculating unit 127, for example, displays all dynamic ranges as indicated by the index 13 in FIG. A combination having the same quantized word length in the number of digits DR is selected. This combination of quantized word lengths is a combination for performing fixed length compression, and a quantized word length having a value smaller by 1 bit than the target code amount is set in all DRs. This is because, in the data format shown in FIG. 8, the bit corresponding to 1 bit / pix is used for the word 0 (min value) in the fixed portion. By setting such a value, the determined code amount always falls within the required code amount.

  When fixed-length compression is performed, a block having a large dynamic range digit number DR generates a large amount of compression noise. However, since the compression parameter is changed for each line, damage to the image when the entire screen is viewed is reduced. Further, since the deterministic code amount can be calculated before the compression process, there is no situation where the code amount exceeds the compression process and the compression fails after the compression process.

  Further, when the quantized word length allocation unit 127 determines the quantized word lengths Q1 to Q10 in the number of digits DR of each dynamic range, the quantized word length allocation unit 127 uses the requested code amount and the determined quantized word lengths Q1 to Q10. The difference from the determined code amount is calculated. This difference is the amount of padding. The quantized word length allocation calculating unit 127 transmits the calculated padding amount to the padding unit 131.

  The quantization word length table shown in FIG. 6 is merely an example, and each pattern described in the quantization word length table is arbitrarily determined based on various factors such as image quality and compression efficiency. It is possible.

  Further, the quantized word length allocation calculation unit 127 can perform compression processing with higher image quality and compression efficiency by further finely branching the quantized word length table according to the frequency DR of the dynamic range. It is possible to select a quantized word length that can be used.

  In addition, the determination of the allocation of the quantization word length Q can be made in addition to the frequency DR of the dynamic range for each block if it can be easily determined which combination should be selected when compressing the block to be compressed. Thus, other image features may be used in combination. For example, it is possible to detect a portion that becomes an edge in a block at the same time as frequency measurement, and to further select the quantization word length Q according to the number of portions that become an edge. For example, when the number of dynamic range digits DR is large and there are one or two decimal edges in one block, the image level changes greatly from dark to bright. The image quality degradation is conspicuous unless a large amount is set. On the other hand, in the case of a striped pattern with many edges even though the number of digits DR of the dynamic range is large, image quality degradation is not noticeable even with a moderate quantization word length. It is easy to measure the number of edges even in maximum / minimum value detection during block compression.

  The quantization execution unit 129 quantizes the image data for each block based on the quantization word lengths Q1 to Q10 calculated by the quantization word length allocation calculation unit 127. Hereinafter, the configuration of the quantization execution unit 129 according to the present embodiment will be described in detail with reference to FIG. FIG. 7 is a block diagram for explaining the configuration of the quantization execution unit 129 according to the present embodiment.

  For example, as illustrated in FIG. 7, the quantization execution unit 129 according to the present embodiment includes a blocking unit 161, a maximum / minimum detection unit 163, a buffer 165, a subtraction unit 167, a dynamic range calculation unit 169, A shift amount calculation unit 171, a quantizer 173, a word length selection unit 175, a quantized data buffer 177, and a packing unit 179 are mainly provided.

  The blocking unit 161 divides the data output from the polygonal line compression unit 131 and delayed for a predetermined period by the one-line delay unit 125 into blocks composed of the same color component pixels for 16 pixels adjacent in the horizontal direction. Output for each block. Thereby, the correlation of the data in a block becomes strong, and the image quality deterioration by subsequent quantization processing can be suppressed.

  The maximum / minimum detection unit 163 detects the maximum value (MAX) and the minimum value (min) from the block data of 16 horizontal pixels for each block. The maximum / minimum detection unit 163 transmits the detected maximum value (MAX) and minimum value (min) to the dynamic range calculation unit 169. Further, the maximum / minimum detection unit 163 transmits the detected minimum value (min) to the subtraction unit 167 and the packing unit 179.

  The buffer 165 holds the data transmitted from the blocking unit 161 for a predetermined period. The buffer 165 holds a block corresponding to input data until the maximum / minimum detection unit 163 calculates the maximum value and the minimum value, and adjusts the delay time.

  The subtraction unit 167 subtracts the minimum value in the corresponding block transmitted by the maximum / minimum detection unit 163 from the pixel data transmitted from the buffer 165. This subtraction is equivalent to subtracting the DC offset common to the compressed pixel area from the data of each pixel.

  The dynamic range calculation unit 169 calculates the difference (MAX−min) between the maximum value (MAX) and the minimum value (min) in one block and calculates the dynamic range in one block. The dynamic range calculation unit 169 transmits the calculated dynamic range to the word length detection unit 171 and the shift amount calculation unit 173.

  The word length selection unit 171 acquires the quantized word lengths Q1 to Q10 transmitted from the quantized word length allocation calculating unit 127, and the quantized word length according to the size of the dynamic range transmitted from the dynamic range calculating unit 169. An appropriate quantization word length Q is selected from Q1 to Q10. Since the quantization word lengths Q1 to Q10 correspond to the dynamic range digits DR = 1 to 10, respectively, quantization suitable for each block is performed according to the size of the dynamic range transmitted from the dynamic range calculation unit 169. It is possible to select the word length Q. The word length selection unit 171 transmits the selected quantized word length Q to the shift amount calculation unit 173 and the packing unit 179.

  The shift amount calculation unit 173 calculates the shift amount using the dynamic range size transmitted from the dynamic range calculation unit 169 and the quantized word length Q transmitted from the word length selection unit 171. Shift amount SFT = dynamic range size DR−quantized word length Q. The shift amount calculation unit 173 transmits the calculated shift amount SFT to the quantizer 175 and the packing unit 179.

  The quantizer 175 quantizes the data transmitted from the subtraction unit 167 according to the shift amount SFT transmitted from the shift amount calculation unit 173. In this embodiment, as an example, quantization is performed to 1 to 6 bits.

  As this quantizer 175, for example, a right shifter can be applied. In this case, the quantizer 175 right-shifts the data transmitted from the subtraction unit 167 by the shift amount SFT transmitted from the shift amount calculation unit 173. By applying a right shifter as the quantizer 175, the circuit scale can be reduced. If the right shifter is used on the compression processing side, the circuit scale can be similarly reduced in the inverse quantization on the decompression processing side.

  The quantized data buffer 177 temporarily holds the quantized data for 16 pixels transmitted from the quantizer 175.

  The packing unit 179 receives the minimum value min transmitted from the maximum / minimum detection unit 163, the shift amount SFT transmitted from the shift amount calculation unit 173, the quantized word length Q transmitted from the word length selection unit 171 and the quantization The 16-pixel quantized data transmitted from the data buffer 177 is packed.

  The packing unit 179 packs these data in a data format as shown in FIG. 8, for example. As shown in FIG. 8, the packed data is provided with a fixed portion in which the quantization word length Q, the minimum value min, and the shift amount SFT are stored at the head, and the quantized data for 16 pixels is provided after the fixed portion. Is provided.

  In the fixed unit in the present embodiment, not only the quantized word length Q but also the minimum value min and the shift are used in order to prevent the dynamic range digit number DR of the block to be compressed from being understood and decoding impossible. The quantity SFT is also stored. By using the quantized word length Q and the shift amount SFT, the number of dynamic range digits (size) DR = quantized word length Q + shift amount SFT can be calculated.

  By executing the quantization process as described above, the data of 16 pixels × 14 bits / pixel = 224 bits can be converted according to the number of digits DR of the dynamic range and while complying with the fixed code amount of one line. This means that the data can be compressed to 32 to 112 bits. This means that a compression ratio that reduces the data amount to about 14% to 50% can be realized.

  Returning to FIG. 2 again, the configuration of the RAW compression unit 101 according to the present embodiment will be described in detail.

  The padding unit 131 adds 0 or 1 to the compressed code data having a variable length (2 to 7 words, 1 word = 16 bits) transmitted from the quantization execution unit 129 and fixed in units of lines. It is a processing unit to be long. In accordance with the amount of padding transmitted from the quantized word length allocation calculating unit 127, the padding unit 131 receives data “0” or data “1” behind the compressed code data transmitted from the quantization executing unit 129. Stuff. As a result, the compressed code data having a variable length in block units and improved compression efficiency has a fixed length in line units.

  The padding unit 131 is composed of, for example, buffers in the horizontal direction total number of pixels × target code amount (bit / pix) × 2 banks. As a result, while the buffer for one bank is used for writing the code to the SDRAM, the other buffer can be used for reading the code from the SDRAM.

  The configuration of the RAW compression unit 101 according to the present embodiment has been described in detail above with reference to FIGS.

  In the above-described example, the case where the fixed-length section in the compression process is 1H (one horizontal synchronization period) has been described. However, the fixed-length section is not limited to the above example. For example, it is possible to flexibly set a length such as H / 4 or 2H based on a length that the system can easily handle or a length that improves efficiency. For example, by setting the fixed-length section to 2H, the possibility of using a portion that has been zero-padded or 1-padded to increase the fixed length in the case of 1H is increased, and compression efficiency can be improved. Become.

[Configuration of RAW decompression unit]
Next, the configuration of the RAW expansion unit 103, which is an example of the expansion unit according to the present invention, will be described in detail with reference to FIGS.

  Note that in the imaging apparatus according to the present embodiment, 16 pixels in the horizontal direction are compressed as one block as described above. In the following, for the sake of simplicity, one block consisting of 16 pixels in the horizontal direction is expressed in units of “block”.

  Further, in the following description, a case where the RAW expansion unit 103 according to the present embodiment expands an arbitrary rectangular area 51 from an image 50 of one frame as illustrated in FIG. 10 will be described. Do. Here, it is assumed that the image 50 of one frame is composed of widthMAX [block] × heightMAX [line] as shown in FIG. Further, as shown in FIG. 10, the rectangular area 51 to be expanded has a size of width [block] × height [line], and is offx [block] in the horizontal direction and offy [line] in the vertical direction from the origin. Suppose they are only separated.

  FIG. 9 is a block diagram for explaining the configuration of the RAW decompression unit according to the present embodiment. As shown in FIG. 9, for example, the RAW expansion unit 103 according to the present embodiment includes a data reading unit 201, a control signal generation unit 203, a head address setting unit 205, a one-block expansion unit 207, and an address holding unit. 209.

  The data reading unit 201 transmits a compression code data read request signal corresponding to an image designated by the user to the SDRAM controller 37, and from the specific address of the corresponding compression code data from the SDRAM 39 through the internal bus 19. Read the starting data. The data reading unit 201 transmits the acquired compressed code data to the 1-block decompression unit 207. In addition, the data reading unit 201 outputs a DEC_START signal to the 1-block decompression unit 207, thereby requesting the 1-block decompression unit 207 to start decompression processing. Further, the data reading unit 201 refers to the data positioned in the first 3 bits (more specifically, the first 3 bits of the fixed part) of the acquired compressed code data, and describes the value of the quantization word length Q described therein. (In other words, the number of words of the variable length part) is transmitted to the address holding unit 209.

  The control signal generation unit 203 has a counter, and is set to 1 in an area that does not need to be expanded based on various setting values (for example, widthMAX, heightMAX, offx, offy, width, height) transmitted from the CPU 27 or the like. The control signal SKIP is generated. The control signal generation unit 203 transmits the SKIP signal that is the generated control signal to the selector 211 provided in the address holding unit 209. Note that the area that does not need to be decompressed means, for example, a portion of the one-frame image 50 shown in FIG. 10 that does not correspond to the rectangular area 51 that is requested to be decompressed.

  Further, the control signal generation unit 203 waits for the end signal BLOCK_DEC_END transmitted from the 1-block expansion unit 207 and performs the next horizontal count-up in the area where data expansion is performed. Further, when the skip or expansion process for one line is completed, the control signal generation unit 203 sets the ADR_INIT signal, which is a control signal, to “1”, and sets the parameter “Tagadr” representing the target address to the next line head address. initialize. The control signal generation unit 203 transmits the generated ADR_INIT signal to the selector 215 included in the head address setting unit 205 and the address holding unit 209. In the present embodiment, since the code amount of one line has a fixed length, the value of the parameter “Tagadr” can be set uniquely.

  The head address setting unit 205 generates a target line head address setting value TAG_INIT_VAL based on various setting values (for example, widthMAX, heightMAX, offx, offy, width, height) transmitted from the CPU 27 or the like. . The start address setting unit 205 transmits the generated address setting value TAG_INIT_VAL to the selector 215 included in the address holding unit 209.

Each time the start address setting unit 205 ends the decompression process (decode) for each line,
The head address of the next line of the compression code data is calculated as follows. That is, since the rectangular area 51 to be expanded is offy [line] and offx [block] away from the origin, the head address setting unit 205 sets “offy × (compression rate × widthMAX)” as an initial value in units of lines. To do. As a result, it is possible to skip the offy line in the vertical direction (the direction along heightMAX in FIG. 10). The start address of the next line may be obtained by sequentially adding “(compression rate × widthMAX)” to the initial value.

  The 1-block decompression unit 207 decompresses the compressed code data transmitted from the data reading unit 201 for each block. The decompression process by the 1-block decompression unit 207 is started when the DEC_START signal output from the data reading unit 201 is received. Hereinafter, the configuration of the 1-block decompression unit 207 according to the present embodiment will be described in detail with reference to FIG. FIG. 11 is a block diagram for explaining the configuration of the 1-block decompression unit 207 according to this embodiment.

  For example, as illustrated in FIG. 11, the 1-block decompression unit 207 according to the present embodiment includes a compression code data latch unit 221, a counter 223, a shift register 225, an inverse quantizer 227, an adder 229, An inverted broken line extending portion 231 is mainly provided.

  When the compression code data latch unit 221 receives the DEC_START signal, the compression code data latch unit 221 latches the compression code data transmitted from the data reading unit 201, and from the compression code data fixing unit (word0), the quantization word length Q, the shift amount SFT, and A value corresponding to the minimum value min is acquired. Further, the compression code data latch unit 221 holds the quantized word length Q as the number of words and words 1 to word 6 as variable length quantized data (code data).

  The compression code data latch unit 221 transmits the code data that is the quantized data and the quantized word length Q to the shift register 225. Further, the compression code data latch unit 221 transmits the held shift amount SFT to the inverse quantizer 227. Further, the compression code data latch unit 221 transmits the held minimum value min to the adder 229.

  When the expansion process is started upon reception of the DEC_START signal, the counter 223 counts the number of pixels subjected to the expansion process. Output a signal. The BLOCK_DEC_END signal output from the counter 223 is transmitted to the control signal generation unit 203.

  The shift register 225 receives data related to the quantization word length Q among the data held in the compression code data latch unit 221. Subsequently, based on the received quantized word length Q, the shift register 225 extracts quantized data for each word length Q from the compressed code data by a shift operation, and transmits the quantized data to the inverse quantizer 227.

  The inverse quantizer 227 inversely quantizes the quantized data for each pixel transmitted from the shift register 225 according to the shift amount SFT transmitted from the compression code data latch unit 221. In the present embodiment, the inverse quantizer 227 performs inverse quantization on the 1 to 6-bit code and outputs 10-bit data.

  As the inverse quantizer 227, for example, a left shifter can be applied. In this case, the inverse quantizer 227 shifts the data transmitted from the shift register 225 to the left by the shift amount SFT transmitted from the compression code data latch unit 221. By applying a left shifter as the inverse quantizer 227, the circuit scale can be reduced.

  The adder 229 adds the minimum value that is a common DC offset value for each block to the code data output from the inverse quantizer 227. This minimum value is transmitted from the compression code data latch unit 221. The adding unit 229 transmits the data added with the minimum value to the reverse polygonal line extending unit 231.

  The reverse broken line expansion unit 231 expands the data transmitted from the addition unit 229 from 10 bits to 14 bits of data with the reverse characteristics of the broken line compression unit 121 of the RAW compression unit 101.

  Here, FIG. 12 is a diagram illustrating an example of a polygonal line used in the reverse polygonal line extending unit 231. The polygonal line in FIG. 12 is adapted to convert the gradation by characteristics opposite to those of the polygonal line compression unit 121 shown in FIG. A ROM table in which input data and decompressed output data are stored in association with each other may be prepared, and input / output data conversion may be performed according to the ROM table. If compression by the polygonal line compression unit 121 is not applied at the time of compression, data conversion at the reverse polygonal line expansion unit 231 is also bypassed at the time of expansion.

  Returning to FIG. 9 again, the configuration of the RAW decompression unit 103 according to the present embodiment will be described in detail.

  The address holding unit 209 calculates an address when reading the designated rectangular area 51 from the compressed code data, and holds the calculated address. For example, as illustrated in FIG. 9, the address holding unit 209 includes selectors 211 and 215, an adder 213, and an access address holding unit 217.

  The selector 211 selects either one of data corresponding to the first 3 bits of the fixed part transmitted from the data reading unit 201 and data representing “1” directly input to the selector 211, The result is output to the adder 213. The selection of data by the selector 211 is performed according to the SKIP signal that is a control signal transmitted from the control signal generation unit 203.

  The adder 213 adds the value transmitted from the selector 211 to the address held by the access address holding unit 217. The adder 213 transmits the value obtained as a result of the addition to the selector 215.

  The selector 215 selects one of the data transmitted from the adder 213 and the setting value TAG_INIT_VAL transmitted from the head address setting unit 205 and outputs the selected value to the access address holding unit 217. The selection of data by the selector 215 is performed according to an ADR_INIT signal that is a control signal transmitted from the control signal generation unit 203.

  The access address holding unit 217 holds the address of the compressed code data read from the SDRAM 39. The data reading unit 201 reads the compressed code data from the SDRAM 39 based on the address held in the access address holding unit 217.

  Next, the flow of setting an access address will be described in detail below.

  First, the access address holding unit 217 is initialized using the set value TAG_INIT_VAL transmitted from the head address setting unit 205, and the data reading unit 201 skips offy [line] in the vertical direction. Here, the value input as a result of the initialization is “TAG_INIT_VAL = offy × widthMax × compression rate [bit / pix]”.

  Subsequently, the address holding unit 209 performs processing for skipping offx block addresses. When the data reading unit 201 reads the data in the SDRAM 39 using the address held in the access address holding unit 217, the data reading unit 201 can access word0 of the block at the head of the line. Here, the data reading unit 201 extracts a 3-bit quantized word length Q according to the format of the compressed code data, and transmits it to the selector 211. The selector 211 transmits the value of the quantized word length Q to the adder 213. The adder 213 adds the quantized word length Q to the address currently held by the access address holding unit 217 and transmits the result to the selector 215. The selector 215 transmits the transmitted value to the access address holding unit 217 as a new address. As a result, the data reading unit 201 can skip to word0 of the address of the next block as shown in FIG.

  As is apparent from FIG. 9, the above-described processing is the operation of the selector 211 (SEL0) tilted to one side and the adder 213. By repeating this process for offx times, it is possible to access from the code data recorded in variable length to the head of the rectangular area 51 required in the line without executing the decompression process. .

  Next, the RAW expansion unit 103 reads out the variable length code unit accompanied by the expansion process. Prior to this processing, the address held in the access address holding unit 217 is a control signal transmitted from the control signal generating unit 203 because it is an address corresponding to the head of the required rectangular area 51. The SKIP signal becomes zero. The data reading unit 201 first reads Word 0 of the block corresponding to the head of the rectangular area 51 held in the access address holding unit 217. Subsequently, the data reading unit 201 reads the compressed code data from the SDRAM 39 by the same amount as the numerical value indicated by the 3-bit quantization word length Q.

  As is apparent from FIG. 9, the above-described processing is the operation of the selector 211 (SEL0) tilted to the 0 side and the adder 213. The data reading unit 201 transmits these (1 + Q) word data to the 1-block decompression unit 207. The 1-block decompression unit 207 decompresses the transmitted compressed code data into a 16-pixel RAW signal. During this time, the counter 223 provided in the 1-block decompression unit 207 waits for “BLOCK_DEC_END = 1” indicating the end of decompression of one block.

  By repeating such processing for width times, it is possible to execute the expansion processing for one line of the desired rectangular area 51.

  Subsequently, the control signal generation unit 203 outputs an ADR_INIT signal, which is a control signal, as preprocessing for performing skip / decompression processing for the next line. As a result, the selector 215 (SEL1) is switched, and the setting value TAG_INIT_VAL transmitted from the head address setting unit 205 is output to the access address holding unit 217. Here, “TAG_INIT_VAL = TAG_INIT_VAL + 1 line skipping” is assumed. The RAW signal decompression unit 103 repeats such processing as high times.

  By performing such processing, the RAW decompression unit 103 according to the present embodiment can decompress an arbitrary rectangular area 51 at high speed without performing unnecessary decompression processing. In order to access the lower block after the expansion process for the number of blocks in the horizontal direction is completed, the address must be updated to indicate the code of the head block of each line. Here, in the imaging apparatus according to the present embodiment, since the compression code data has a fixed length in units of lines, the address can be uniquely determined.

  In the above-described example, the case where the arbitrary rectangular area 51 is read and expanded for each line has been described. However, for example, as illustrated in FIG. It is also possible to perform expansion processing in a tile shape. Here, the tile 53 has a size of a block to be expanded × a predetermined number of lines.

  When the rectangular area 51 as shown in FIG. 13 is expanded in a tile shape, the skipping process up to the expansion start position can be performed in the same manner as described above in which the expansion process is performed for each line. At this time, the access address holding unit 217 has data holding means such as SDRAM for the number of lines included in one tile 53, and holds the address of the variable length code being expanded in each line. It is necessary to keep.

  By performing such processing, the tile 53 located on the right side can be expanded in order. Since the size of the tile as a unit of normal processing is small, even if the access address holding unit 217 is held for the number of lines, the scale can be small.

  Heretofore, an example of the function of the imaging device 10 according to the present embodiment has been shown. Each component described above may be configured using a general-purpose member or circuit, or may be configured by hardware specialized for the function of each component. In addition, the CPU or the like may perform all functions of each component. Therefore, it is possible to appropriately change the configuration to be used according to the technical level at the time of carrying out the present embodiment.

<Summary>
[Summary about compression processing]
As described above, the RAW compression unit 101 according to the present embodiment performs variable-length quantization in units of blocks and compresses 14-bit / pix data to 6-bit / pix. This
It is possible to reduce the BUS bandwidth and the memory area (and thus improve the number of continuous shots) during RAW capture. In addition, since the padding unit 131 performs padding processing on the data “0” or the data “1”, the compression code data has a fixed length in units of lines. As a result, the compressed code data has a fixed length even in frame units.

  Further, the RAW compression unit 101 according to the present embodiment does not use predictive coding and variable length codes. Therefore, the decompression process can be performed with a simple calculation. Further, since the correlation between lines is not used, it is possible to deal with not only CMOS but also CCD that requires field readout.

  Also, in the case of the compression method using the conventional variable length compression, there is a possibility that the code amount will eventually exceed the original image size, so a mechanism for detecting and processing the situation where the code amount is exceeded is necessary. Met. Specifically, such a mechanism has conventionally been realized as an interrupt signal generation circuit to the CPU or an interrupt processing program that executes processing for discarding compressed data and displaying a shooting failure to the user. . However, in the compression method according to the present embodiment, since the code amount does not eventually exceed the target code amount, such a mechanism is unnecessary and the control is simple.

  Further, in the compression method according to the present embodiment, the code amount of one line is determined instead of the estimated value when determining the allocation of the quantization word length while ensuring the real-time property for capturing RAW data at the time of shooting. Can be obtained by value. For this reason, although the variable length element is included in the quantization, the generated code can always be a fixed length. Also, the compression method according to the present embodiment is different from the method of compressing by changing to the fixed length method after monitoring the code amount while compressing by the variable length method and exceeding the setting. Since the fixed length is used when the amount is exceeded, the damage to the entire screen is reduced.

[Summary about decompression processing]
As described above, since the RAW compression unit 101 according to the present embodiment has a fixed length in units of frames, any frame that has been continuously captured in the memory by continuous shooting at the time of expansion processing. Random access to is possible. In the present embodiment, since the code format is adopted such that the quantization word length + 1 (or a fixed value) in the compression code data of one block is the number of words of the compression code data, the code length is The data to be represented is not particularly required in the compression code. Thereby, it is possible to improve the compression efficiency.

  The RAW decompression unit 103 according to the present embodiment reads a stream of one line of fixed length into a memory during decompression processing, sequentially searches the first word from the head, acquires only the quantized word length, and adds it to the address. go. Thereby, the imaging apparatus according to the present embodiment can access the first word of the target block without performing the expansion process.

  This means that it is possible to access a desired block in one frame without expanding the entire frame, and E-zoom and trimming that require extraction of an arbitrary rectangular area in one frame. Thus, it is possible to reduce processing time and memory.

  Conventionally, when a variable length code is used to improve compression efficiency, the decompression process must be performed from the beginning of one frame. This is because if there is a break in the code every several blocks and several lines, it will be necessary to insert a code that will be a code truncation mark or record the length to the next coding unit, etc. This is because it is difficult to introduce from the aspect of improving the compression efficiency. This means that if you want to access a desired block, you must prepare a buffer for one frame, expand the data once to this buffer, and then access the desired rectangular area. To do. Such processing leads to expansion of the memory area and increase in processing time.

  However, in the compression / decompression method according to this embodiment, although the length is variable in units of blocks, the length is fixed in units of lines, so that random access to any frame captured on the memory is performed. Is possible. Thereby, it is possible to suppress expansion of the memory area and increase in processing time.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this example. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

It is a block diagram for demonstrating the structure of the imaging device which is an example of the information processing apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram for demonstrating the structure of the compression part which concerns on the same embodiment. It is explanatory drawing for demonstrating a Bayer arrangement | sequence and the extracted same color 1 line. It is a graph for demonstrating the example of the broken line used in the broken line compression part which concerns on the same embodiment. It is a block diagram for demonstrating the structure of the frequency measurement part which concerns on the same embodiment. It is explanatory drawing which showed an example of the quantization word length table which the quantization word length allocation calculation part which concerns on the embodiment refers. It is a block diagram for demonstrating the structure of the quantization execution part which concerns on the same embodiment. It is explanatory drawing for demonstrating the data structure of the compressed image data which concerns on the embodiment. It is a block diagram for demonstrating the structure of the expansion | extension part which concerns on the same embodiment. It is explanatory drawing for demonstrating the parameter used in order to specify the data reading position in compressed image data. It is a block diagram for demonstrating the structure of the 1-block expansion | extension part which concerns on the same embodiment. It is a graph for demonstrating an example of the conversion characteristic in the reverse broken line conversion part which concerns on the same embodiment. It is explanatory drawing for demonstrating an example of the data reading method in compressed image data. It is a block diagram for demonstrating the structure of the conventional imaging device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up device 11 Image sensor 13 AFE circuit part 15 Digital image processing circuit part 17 Camera signal pre-processing part 19 Internal bus 21 Camera signal processing part 23 Resolution conversion part 25 JPEG engine 27 CPU
29 ROM
31 Storage Device 33 Video Output Encoder 35 Video Output Terminal 37 SDRAM Controller 39 SDRAM
50 1-frame image 51 Rectangular area 53 Block 101 RAW compression unit 103 RAW expansion unit

Claims (9)

A compression unit that compresses image data output from the image sensor;
A storage unit for holding the compressed image data;
Have
The compression unit is
For the image data, a frequency measurement unit that measures the frequency of the dynamic range of a block composed of a predetermined number of pixel data;
Based on the frequency, for each block, a quantized word length allocation calculating unit that calculates a quantized word length allocated to each dynamic range;
A padding unit for adding 0 or 1 to a compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length until a predetermined code amount is reached; ,
An image processing apparatus comprising: The image processing apparatus holds in advance a quantization word length table in which a plurality of patterns having different combinations of quantization word lengths assigned to the respective dynamic ranges are described,
The quantized word length allocation calculating unit calculates a determined code amount of compressed image data for the predetermined block in each pattern based on the plurality of patterns and the frequency described in the quantized word length table, The image processing apparatus according to claim 1, wherein a combination of the quantization word lengths in the pattern in which the determined code amount is equal to or less than a desired code amount is selected.   The quantization word length allocation calculation unit sets a quantization word length allocated to each dynamic range to a predetermined fixed value when the determined code amount exceeds the desired code amount. The image processing apparatus described.   The quantized word length allocation calculating unit transmits a difference between the predetermined code amount and the determined code amount in the selected combination of quantized word lengths as a padding amount to the padding unit. An image processing apparatus according to 1. The compression unit further includes a quantization execution unit that quantizes the image data for each block based on the calculated quantization word length,
The quantization execution unit has a predetermined number of bits, a fixed-length unit including at least information indicating the calculated quantization word length, and a variable including the encoded data corresponding to the predetermined number of pixel data The image processing apparatus according to claim 1, wherein the encoded data includes a long part. The image processing apparatus further includes a decompression unit that obtains the compressed image data held in the storage unit and decompresses data corresponding to an arbitrary rectangular area from the compressed image data,
The image processing apparatus according to claim 5, wherein the decompressing unit specifies a position of data corresponding to the arbitrary rectangular area based on information representing the quantization word length included in the fixed length unit. Measuring the frequency of the dynamic range of a block composed of a predetermined number of pixel data for the image data output from the image sensor;
Calculating a quantized word length assigned to each dynamic range for each block based on the frequency;
Adding 0 or 1 to compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length until a predetermined code amount is reached;
Including an image processing method. The image data output from the image sensor is input, and the computer can compress and hold the input image data.
For the input image data, a frequency measurement function for measuring the frequency of the dynamic range of a block consisting of a predetermined number of pixel data;
Based on the frequency, for each block, a quantization word length allocation calculation function for calculating a quantization word length allocated to each dynamic range;
A padding function for adding 0 or 1 to the compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length until a predetermined code amount is reached; ,
A program to realize A compression unit that compresses image data output from the image sensor;
A storage unit for holding the compressed image data;
Have
The compression unit is
For the image data, a frequency measurement unit that measures the frequency of the dynamic range of a block composed of a predetermined number of pixel data;
Based on the frequency, for each block, a quantized word length allocation calculating unit that calculates a quantized word length allocated to each dynamic range;
A padding unit that adds 0 or 1 to compressed image data including a predetermined block of encoded data for each block quantized based on the calculated quantized word length until a predetermined code amount is reached; ,
An imaging apparatus comprising:

Images