JP4100836B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention,Image processing apparatus for processing image data obtained by an image sensorIn placeaboutThe
[0002]
[Prior art]
When compressing and recording images or arbitrary input data, the input data series is subjected to orthogonal transform such as discrete cosine transform (DCT) or DPCM (Differential Pulse Modulation Modulation) to reduce the entropy of the input data series. The input data is compressed by entropy coding represented by Huffman code or arithmetic code.
[0003]
Huffman coding achieves data compression by assigning short codewords to data with high appearance frequency and long codewords to data with low appearance frequency from the histogram of the input data series, and reducing the data size on average. .
[0004]
In the JPEG baseline method using DCT and Huffman coding, coding processing is performed according to a set Huffman table. All information necessary for encoding is described in the Huffman table, and the description content is generated from the histogram of the data series to be encoded.
[0005]
[Problems to be solved by the invention]
In order to realize the most effective encoding in Huffman encoding, it is necessary to calculate a histogram of a data sequence to be encoded and generate a unique Huffman table corresponding to each data sequence.
[0006]
However, in the image compression / decompression of a digital camera (electronic camera) or the like, a single Huffman table provided as a standard is used because of problems such as processing speed. Therefore, effective encoding (compression) is not always realized.
[0007]
  The present inventionImage data (for example, CCD-RAW Data)Image processing device that realizes effective codingPlaceThe purpose is to provide.
[0008]
In the image processing apparatus according to the present invention, in the case where the output signal of the image sensor is digitized and converted into first image data, and the number of bits of the first image data is the first number of bits The first image data is encoded according to a first encoding method, and the number of bits of the first image data is larger than the first number of bits.First 2 Is the number of bitsIn this case, the first image data is divided into first partial data for the first number of bits and second partial data having a portion other than the first partial data, and the first Encoding means for encoding the partial data according to the first encoding method, and storing the first image data on a recording medium, and the number of bits of the first image data is the first When the number of bits is 1, the first image data encoded by the encoding means is stored in the recording medium, the first image data is stored in the recording medium, And the number of bits of the first image data isFirst 2 Is the number of bitsIn some cases, the recording apparatus may include recording means for storing the first partial data encoded by the encoding means and the second partial data in the recording medium.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.
[0014]
[Constitution]
FIG. 1 is a block diagram showing a configuration example of an image processing apparatus 1000 according to an embodiment of the present invention.
[0015]
An image of the object to be photographed is optically formed on an image pickup device 12 such as a CCD that converts the photographed image into an electric signal optically by the photographing lens 10. The analog output signal of the image sensor 12 is converted into a digital signal by the A / D converter 14. In the following, image data output from the A / D converter 14 is referred to as CCD-RAW data in order to distinguish it from image-processed image data.
[0016]
The memory control circuit 40 controls the data flow in the A / D converter 14 and the D / A converter 20, image processing circuit 50, memory 60, recording medium 70, JPEG circuit 80, and data conversion circuit 100, which will be described later. . The memory control circuit 40 includes a plurality of memory control circuits dedicated to writing data into the memory 60 and a plurality of memory control circuits dedicated to reading data from the memory 60.
[0017]
The image processing circuit 50 performs pixel interpolation processing and color conversion processing on the CCD-RAW data having the element array (for example, Bayer array) as shown in FIG. Form.
[0018]
The memory 60 is a memory for storing captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. The CCD-RAW data output from the A / D converter 14 is written into the memory 60 via the memory control circuit 40 and the image processing circuit 50, or directly from the memory control circuit 40. Further, the display image data formed by the image processing circuit 50 from the CCD-RAW data and written in the memory 60 is transferred to the image display unit such as a TFT LCD via the memory control circuit 40 and the D / A converter 20. Displayed at 22.
[0019]
The recording medium 70 is a semiconductor memory card, a magnetic recording medium floppy disk or hard disk, or a magneto-optical disk, and a removable medium is preferably used.
[0020]
  JPEGThe circuit 80 includes a DCT / quantization circuit 82, a data selector 84, and a Huffman encoding / decoding circuit 86, and compresses / decompresses image data by the baseline JPEG method. The data selector 84 switches the data flow according to JPEG encoding / decoding of image data and lossless compression / decompression of CCD-RAW data, that is, according to a recording mode described later.
[0021]
The data conversion circuit 100 is a circuit that performs data conversion for lossless compression / decompression of CCD-RAW data using the Huffman encoding / decoding circuit 86 of the JPEG circuit 80, and includes an interface circuit 102, a division / synthesis circuit 106. And a data conversion core 104 including a DPCM conversion circuit 108. The data conversion core circuit 104 is a part that actually performs the data conversion processing in the present embodiment.
[0022]
The interface circuit 102 is a circuit for the data conversion circuit 100 to perform data handshake with other circuit blocks (for example, the memory control circuit 40 and the JPEG circuit 80). At the same time, the design of 104 becomes easy, and the development of software for controlling the data transfer of the entire system becomes easy. Detailed configuration and operation will be described later.
[0023]
The Huffman encoding / decoding circuit 86 is a circuit designed for baseline JPEG, and the data bus between the data selector 84 and the Huffman encoding / decoding circuit 86 is 11 bits. Therefore, CCD-RAW data to be DPCM converted must be 10 bits or less.
[0024]
When the CCD-RAW data is 12 bits at the time of compression, the dividing / combining circuit 106 divides the data into upper 10 bits and lower 2 bits, and performs PACK processing on the lower 2 bits every 8 data. If the CCD-RAW data is 12 bits at the time of decompression, the packed lower 2 bits of data are subjected to UNPACK (unpacking) processing, and the DPCM conversion circuit 108 combines them with 10-bit data subjected to inverse DPCM conversion. FIG. 2 shows data formats in the division and PACK processing, and the UNPACK and synthesis processing.
[0025]
The DPCM conversion circuit 108 performs a DPCM conversion (predictive encoding) of CCD-RAW data in order to reduce the entropy of information and increase the encoding efficiency in Huffman encoding. The DPCM conversion circuit 108 performs DPCM conversion (predictive coding) on 10-bit data and performs inverse DPCM conversion on 11-bit DPCM data. The DPCM conversion reduces the entropy of information by utilizing the fact that the image information of the pixel of interest to be encoded and the image information of surrounding pixels are strongly correlated. Specifically, the encoding efficiency in the Huffman encoding is increased by converting the image data of the target pixel into a difference value with the image data of the adjacent pixel (left adjacent pixel).
[0026]
This embodiment corresponds to the image sensor 12 having the color filter array of the element array as shown in FIG. 3, and it is necessary to always calculate a difference value from the CCD-RAW data adjacent to the left of two pixels. That is, the DPCM conversion circuit 108 is configured to obtain a difference between the input CCD-RAW data and the CCD-RAW data input two pixels before. Since prediction in DPCM conversion is easier for low-frequency components of an image, when CCD-RAW data is 12 bits, only the upper 10 bits are DPCM-converted and high-frequency components that are difficult to predict are recorded uncompressed. More detailed configuration and operation will be described later.
[0027]
The data selector 90 switches the data flow according to JPEG encoding / decoding and when performing lossless compression / decompression of CCD-RAW data, that is, according to a recording mode described later.
[0028]
The system control circuit 30 includes a CPU, a RAM, a ROM, and the like. According to the program stored in the ROM, the setting of the mode dial 32 and the recording mode switch 34, and the recorded contents of the read-only memory (ROM) 36, The operation of the entire image processing apparatus 1000 and each circuit block is controlled.
[0029]
The mode dial 32 is used by the user to switch function modes of the image processing apparatus 1000 such as power on / off, shooting mode, and playback mode. The recording mode switch 34 is for the user to select either the lossless compression / decompression mode of CCD-RAW data or the JPEG recording mode. Note that when the user selects the JPEG recording mode, the compression rate and the like can also be set by the recording mode switch 34. In the read-only memory 36, a program executed by the system control circuit 30, a quantization table and a Huffman table set in the JPEG circuit 80, and the like are recorded.
[0030]
[Data conversion core]
FIG. 4 is a block diagram illustrating a detailed configuration example of the data conversion core 104. In FIG. 4, the part other than the DPCM conversion circuit 108 corresponds to the dividing / combining circuit 106.
[0031]
A pair of VALID signals and STOP signals exist in all data series input to and output from the data conversion core 104. When the VALID signal is ‘1’, the input / output data is valid, and when the STOP signal is ‘1’, the data input / output is invalid. The data conversion core 104 performs processing assuming that valid data is input / output when the VALID signal accompanying the input / output data is ‘1’ and the STOP signal is ‘0’. The following six types of data series are input to and output from the data conversion core 104.
CCD_IN_DATA: CCD-RAW data to be compressed (from memory control circuit 40)
DPCM_OUT_DATA: DPCM converted data (to JPEG circuit 80)
PACK_DATA: Packed lower 2 bits data (to memory control circuit 40)
CCD_OUT_DATA: Expanded CCD-RAW data (to memory control circuit 40)
DPCM_IN_DATA: Data to be subjected to inverse DPCM conversion (from JPEG circuit 80)
UNPACK_DATA: Data to be unpacked
[0032]
A VALID signal (input) and a STOP signal (output) are attached to each input data series, and a VALID signal (output) and a STOP signal (input) are attached to each output data series.
[0033]
There are three types of STOP signals, DPCM_OUT_STOP, PACK_STOP, and CCD_OUT_STOP, which are input to the data conversion core 104 in association with the output data of the data conversion core 104.
[0034]
These blocks having the STOP signal set to “1” mean that the data output from the data conversion core 104 cannot be received, and the output data of the data conversion core 104 becomes invalid. Therefore, when any of these STOP signals is “1” and the corresponding VALID signal is “1”, all the flip-flops (buffers) in the data conversion core 104 are in the holding state, and all the STOP signals are “ No data is updated until it reaches 0 '.
[0035]
On the other hand, accompanying the input data of the data conversion core 104, there are three types of STOP signals output from the data conversion core 104: CCD_IN_STOP, DPCM_IN_STOP, and UNPACK_STOP.
[0036]
When these STOP signals are “1”, it means that the data conversion core 104 cannot receive input data, and the input data from the block to which the STOP signals are connected is not updated until the STOP signal becomes “0”. Therefore, the control signal generator 180 or 182 of the data conversion core 104 delays the next data input according to the operation state in the data conversion core 104, or receives data from a plurality of blocks and waits for the data. In this case, the update of data input to the data conversion core 104 can be temporarily interrupted by setting the STOP signal sent to the corresponding block to “1”.
[0037]
● Lossless compression of CCD-RAW data
The operation of the data conversion core 104 when reversibly compressing CCD-RAW data is as follows.
[0038]
The CCD-RAW data (CCD_IN_DATA) to be compressed is read into the buffer 184 when the VALID signal (CCD_IN_VALID) is '1', and the CCD-RAW data is read into the buffer 184 when the VALID signal (CCD_IN_VALID) is '0'. RAW data is retained. An example of the data format of the CCD-RAW data held in the buffer 184 is shown in FIG. Regardless of the data width of the CCD-RAW data, 10-bit data from the 11th bit to the 2nd bit of the data held in the buffer 184 is transferred to the DPCM conversion circuit 108, and DPCM conversion is performed. The DPCM-converted data is once held in the buffer 190 and then output from the data conversion core 104. The control signal generator 180 creates a VALID signal (DPCM_OUT_VALID) corresponding to the DPCM converted data from the input VALID signal (CCD_IN_VALID), and outputs the VALID signal according to the DPCM converted data.
[0039]
When the data width of the CCD-RAW data is 12 bits, the lower 2 bits are separated and the PACK process is performed according to the format shown in FIG. The lower 2 bits of the data held in the buffer 184 are held in any of the eight buffers 200 to 214. The control signal generator 180 counts the input VALID signal (CCD_IN_VALID) to determine what number the data stored in the buffer 184 is input, and based on the determination result, An 8-bit control signal for selecting one of 200 to 214 is generated. A buffer whose corresponding bit of the 8-bit control signal is “1” is updated by reading 2-bit data from the buffer 184. On the other hand, a buffer whose corresponding bit of the 8-bit control signal is ‘0’ holds data.
[0040]
The outputs of the buffers 200 to 214 are connected to the buffer 192. The data in the buffer 192 is updated every 8 input data by the control signal of the control signal generator 180, and the updated data is output as packed data. At this time, the control signal generator 180 outputs a “1” VALID signal (PACK_VALID) corresponding to the PACK data.
[0041]
● Decompression of CCD-RAW data
The operation of the data conversion core 104 when expanding the CCD-RAW data is as follows.
[0042]
DPCM data (DPCM_IN_DATA) to be subjected to inverse DPCM conversion is read into the buffer 188 when the VALID signal (DPCM_IN_VALID) is '1', and is read into the buffer 188 when the VALID signal (DPCM_IN_VALID) is '0'. Is retained. The DPCM data held in the buffer 188 is subjected to inverse DPCM conversion by the DPCM conversion circuit 108, and the inversely converted 10-bit data is input from the 11th bit to the 2nd bit of the buffer 194. When the original data width of the compressed and recorded CCD-RAW data is 10 bits, “00” corresponding to the lower 2 bits of the buffer 194 is input to the buffer 194 via the multiplexer 198 as dummy data. The output of the buffer 194 is output as decompressed CCD-RAW data (CCD_OUT_DATA). The control signal generator 182 creates a VALID signal (CCD_OUT_VALID) for the decompressed data from the input VALID signal (DPCM_IN_VALID), and outputs it according to the CCD-RAW data.
[0043]
When the original data width of the compressed and recorded CCD-RAW data is 12 bits, UNPACK and synthesis processing of lower 2 bits are performed according to the format shown in FIG. Data to be unpacked (UNPACK_DATA) is read into the buffer 186 when the VALID signal (UNPACK_VALID) is “1”, and the data read into the buffer 186 is held when the VALID signal (UNPACK_VALID) is “0”. The data held in the buffer 186 is read by the multiplexer 196 two bits at a time, and is connected to the lower two bits of the buffer 194 via the multiplexer 198.
[0044]
Which 2 bits are selected by the multiplexer 196 is determined by the 4-bit control signal of the control signal generator 182. The control signal generator 182 counts the input VALID signal (DPCM_IN_VALID), thereby determining what number the 10-bit data input to the buffer 194 is, and generates a 4-bit control signal. The control of whether the multiplexer 198 selects the output of the multiplexer 196 or the dummy data “00” is also performed by the 4-bit control signal.
[0045]
The 12-bit data synthesized in the buffer 194 is output as decompressed CCD-RAW data (CCD_OUT_DATA). The control signal generator 182 creates a VALID signal (CCD_OUT_VALID) corresponding to the decompressed data from the input VALID signal (DPCM_IN_VALID), and outputs it according to the CCD-RAW data.
[0046]
Further, the data in the buffer 186 is updated only once for every eight data subjected to inverse DPCM conversion. Therefore, the control signal generator 182 creates and outputs the STOP signal (UNPACK_STOP) while counting the input VALID signal (DPCM_IN_VALID), thereby waiting for the data of the upper 10 bits and the lower 2 bits. Easy to do.
[0047]
● Waiting process
FIG. 6 is a timing chart showing a specific example of the waiting process.
[0048]
Hereinafter, in order to simplify the description, the data sent from the DPCM conversion circuit 108 to the buffer 254 will be described as 10bit_DATA, and the VALID signal and STOP signal accompanying 10bit_DATA will be described as 10bit_VALID and 10bit_STOP, respectively. Actually, the control signal generator 182 performs control in consideration of the data delay of the DPCM conversion circuit 108.
[0049]
At timing t1, since the UNPACK_VALID signal is “1” and the UNPACK_STOP signal is “0”, P1 which is UNPACK_DATA is read into the buffer 186. Since UNPACK_DATA is updated only twice for eight 10bit_DATA, the control signal generator 182 sets the UNPACK_STOP signal to ‘1’ so that it is not updated to P2 of UNPACK_DATA at the next timing t2.
[0050]
At timing t2, since the 10bit_VALID signal is “1”, the selected data D1 and P1 are combined by the buffer 194 and output as CCD_OUT_DATA. The control signal generator 182 sets the CCD_OUT_VALID signal to “1”.
[0051]
At timing t8, the selected data D7 and P1 are combined by the buffer 194 and output as CCD_OUT_DATA. Since the process of synthesizing the seven data is completed by the process at timing t8, it is necessary to update the data in the buffer 186 after the process of synthesizing the eighth data is completed at the next timing t9. Therefore, at timing t8, the control signal generator 182 changes the UNPACK_STOP signal from “1” to “0”.
[0052]
At timing t9, the selected data D8 and P1 are combined by the buffer 194 and output as CCD_OUT_DATA. On the other hand, P2 of UNPACK_DATA is read into the buffer 186. Then, similarly to the timing t1, the control signal generator 182 sets the UNPACK_STOP signal to “1”.
[0053]
At timing t10, the selected data D9 and P2 are combined by the buffer 194 and output as CCD_OUT_DATA. Further, since the UNPACK_VALID signal is “0”, there is no need to issue a STOP signal, so the control signal generator 182 sets the UNPACK_STOP signal to “0”.
[0054]
At timing t17, the selected data D16 and P2 are combined by the buffer 194 and output as CCD_OUT_DATA. Since the process of synthesizing the eight data D9 to D16 is completed by the process at timing t17, the data in the buffer 186 needs to be updated. However, the UNPACK_VALID signal is “0” and UNPACK_DATA is undefined. Therefore, since the synthesis process cannot be performed at the next timing t18, the control signal generator 182 sets the 10bit_STOP signal to ‘1’ so that the 10bit_DATA is not updated.
[0055]
At timing t18, since the 10bit_STOP signal is “1”, 10bit_DATA is not updated as D17. Since the UNPACK_VALID signal is “1”, P3 of UNPACK_DATA is read into the buffer 186. Similar to the timing t1, the control signal generator 182 sets the UNPACK_STOP signal to ‘1’.
[0056]
At timing t19, the selected data D17 and P3 are combined by the buffer 194 and output as CCD_OUT_DATA.
[0057]
In this way, the data conversion core 104 easily realizes a waiting process by outputting a STOP signal as necessary. In addition, since the data conversion core 104 implements the waiting process in hardware, the system control circuit 30 hardly needs to control the data transfer between the circuit blocks.
[0058]
[Interface circuit]
As described above, the data conversion core 104 easily realizes a waiting process when receiving data from two blocks by the VALID signal and the STOP signal. However, in order to actually perform data handshake with other blocks, an interface circuit 102 for controlling the VALID signal and the STOP signal is required around the data conversion core 104.
[0059]
FIG. 7 is a block diagram showing a most basic configuration example of the interface circuit 102. As shown in FIG.
[0060]
An arithmetic unit 302 corresponding to the data conversion circuit 100 includes an interface circuit 102 that performs data handshaking with the arithmetic units 300 and 304, and an arithmetic circuit 308 corresponding to the data conversion core 104 that processes input data. The The plurality of arithmetic units constituting the system is based on the configuration of the arithmetic unit 302, and all blocks are designed with the same configuration.
[0061]
Every data series that propagates between units is accompanied by a pair of VALID and STOP signals for each data series. All data transfer between each arithmetic unit in the apparatus is controlled by these signals.
[0062]
The VALID signal indicates that the data series transferred between the arithmetic units is valid data at a certain timing, and is propagated in the same direction as the propagation direction of the data series. In the present embodiment, a cycle in which the VALID signal is “1” means that the data series is valid, and only one valid data series is output.
[0063]
The STOP signal instructs to stop updating the data of the input data series when the data input at the next timing cannot be processed because the arithmetic unit that should receive the data is processing the data. Is transmitted in the direction opposite to the propagation direction of the data series. In the present embodiment, a cycle in which the STOP signal is “1” means that the arithmetic unit cannot process data.
[0064]
As described above, the purpose of the interface circuit 102 is to facilitate the design of the arithmetic circuit 308 that performs data processing and to facilitate control of the entire system.
[0065]
The arithmetic circuit 308 receives data and performs necessary data processing only when the input VALID_IN signal is ‘1’ at a certain timing. Further, when valid data transfer is performed to the arithmetic unit 304 connected to the subsequent stage, the VALID_OUT signal is set to ‘1’. However, even if the VALID_OUT signal is '1', if the OUT_STOP signal, which is a STOP signal from the arithmetic unit 304 connected to the subsequent stage, is '1', it is a STOP signal to the arithmetic circuit 308. The STOP_OUT signal becomes '1'. That is, since the arithmetic unit 302 cannot update data, all internal states in the arithmetic circuit 308 are held. Further, when it is desired to stop data transfer from the arithmetic unit 300 connected to the preceding stage of the arithmetic unit 302, the STOP_IN signal is set to ‘1’. That is, the arithmetic circuit 308 takes in valid data only when the VALID_IN signal is ‘1’ and the STOP_IN signal and the STOP_OUT signal are ‘0’.
[0066]
In actual data processing, most control can be performed simply by counting the VALID signal input to the arithmetic circuit 308.
[0067]
The interface circuit 102 is located around the arithmetic circuit 308 that performs actual data processing, and performs data handshaking with an adjacent arithmetic unit. The basic configuration of the interface circuit 102 is a buffer 110 composed of n D flip-flops corresponding to the bus width (n bits) of IN_DATA that is input data, one D flip-flop, and n + 1 bit output Data selector 114 and a plurality of logic gates.
[0068]
[Function of interface circuit]
8A and 8B are timing charts showing in detail the relationship among data, the VALID signal, and the STOP signal, and explain the function of the interface circuit 102. FIG. Note that the operations of the arithmetic circuit 308, the arithmetic unit 300, and the arithmetic unit 304 in the present embodiment are merely assumed for explanation. 8A and 8B are a series of timing charts.
[0069]
At timing t0, the output of the D-FF 112 is reset to “0” by the reset signal RESET. As a result, the IN_STOP signal output from the arithmetic unit 302 becomes “0”. Since the selector 114 is set on the data input side (non-buffer 110 side), the IN_DATA and IN_VALID signals are directly connected to the DATA_IN and VALID_IN signals of the arithmetic circuit 308.
[0070]
At timing t1, DATA_IN as input data is indefinite, and the VALID_IN signal input to the arithmetic circuit 308 is “0”, so the arithmetic unit 302 does not perform any processing. Immediately after t1, IN_DATA, which is input data from the arithmetic unit 300 connected in the previous stage, is determined and the IN_VALID signal changes to '1', so the VALID_IN signal input to the arithmetic circuit 308 changes to '1' .
[0071]
At timings t2 and t3, the VALID_IN signal input to the arithmetic circuit 308 is '1', and the STOP_IN signal and the STOP_OUT signal are '0'. It is read into the arithmetic circuit 308 and processed.
[0072]
At timing t4, the VALID_IN signal input to the arithmetic circuit 308 is “1”, but the STOP_IN signal output from the arithmetic circuit 308 indicates “1”. That is, since the arithmetic circuit 308 is currently processing data, it means that the next data cannot be received. In this case, the output of the logic gate 118, which is the logical product of the VALID_IN signal and the STOP_IN signal, sets the D-FF112 to '1' and outputs the IN_STOP signal of '1' to the arithmetic unit 300. Set to 110 side. The IN_STOP signal changes the IN_VALID signal from ‘1’ to ‘0’ by the same operation as the logic gate 124 described above. The outputs of the logic gate 118 and the D-FF 112 become the LOAD signal (LD) of the buffer 110 via the logic gate 116. Accordingly, at timing t4, D3 of IN_DATA is read into the buffer 110, and the read data D3 * is output as DATA_IN to the arithmetic circuit 308.
[0073]
At timing t5, since the IN_STOP signal output from the arithmetic unit 302 is “1”, IN_DATA output from the arithmetic unit 300 is not updated. Further, since the VALID_IN signal input to the arithmetic circuit 308 and the STOP_IN signal output from the arithmetic circuit 308 remain “1”, the D-FF 112 is continuously set to “1”. Also, the buffer 110 is not updated.
[0074]
At timing t6, since the IN_STOP signal output from the arithmetic unit 302 is “1”, IN_DATA output from the arithmetic unit 300 is not updated. However, since the VALID_IN signal input to the arithmetic circuit 308 remains “1” and the STOP_IN signal output from the arithmetic circuit 308 becomes “0”, the data D3 * of the buffer 110 connected to DATA_IN is calculated. It is read into the circuit 308 and processed. The STOP_IN signal resets the D-FF 112 to ‘0’ via the logic gate 118, and sets the IN_STOP signal output from the arithmetic unit 302 to ‘0’. Since the selector 114 selects the data input side, DATA_IN of the arithmetic circuit 308 is directly connected to IN_DATA of the arithmetic unit 302 again. Further, the IN_STOP signal changes the IN_VALID signal from ‘0’ to ‘1’ by the same operation as the gate circuit 124 described above.
[0075]
At timing t7, the VALID_IN signal input to the arithmetic circuit 308 is “1”, and the STOP_IN signal output from the arithmetic circuit 308 and the STOP_OUT signal input to the arithmetic circuit 308 are “0”. D4 is read into the arithmetic circuit 308 as valid data via the selector 114 and processed. Further, since the IN_STOP signal output from the arithmetic unit 302 is “0”, the data output from the arithmetic unit 300 is updated.
[0076]
At timing t8, as in timing t7, D5 of IN_DATA is processed by the arithmetic circuit 308, and IN_DATA is updated.
[0077]
At the timing t9, since the IN_VALID signal input to the arithmetic unit 304 is “0”, IN_DATA means invalid (undefined). Although IN_DATA is connected to DATA_IN of the arithmetic circuit 308, the arithmetic circuit 308 does not perform processing because it is invalid data. In this embodiment, it is assumed that IN_DATA is indefinite. However, as described above, the IN_VALID signal is designed to be '0' when IN_DATA from the arithmetic unit 300 remains D5 and is not updated. .
[0078]
At timing t10, IN_DATA input to the arithmetic unit 302 is ignored in the same manner as at timing t9. On the other hand, since the VALID_OUT signal output from the arithmetic circuit 308 is ‘1’ and the OUT_STOP signal input to the arithmetic unit 304 is ‘0’, ‘1’ is output as the OUT_VALID signal. D1 of DATA_OUT is transferred to the arithmetic unit 304 via OUT_DATA, and is processed as valid data in the arithmetic unit 304.
[0079]
At timings t11 and t12, similarly to timing t7, D6 and D7 of IN_DATA input from the arithmetic unit 300 are processed by the arithmetic circuit 308, and the data is updated. Similarly to timing t10, d2 and d3 of OUT_DATA output from the arithmetic circuit 308 are processed by the arithmetic unit 304, and the data is updated. Further, since the arithmetic unit 304 cannot receive new data at timing t13 immediately after receiving d3 of OUT_DATA, the OUT_STOP signal output to the arithmetic unit 302 is set to ‘1’. This is the same as the process in which the IN_STOP signal output from the arithmetic unit 302 to the arithmetic unit 300 is changed from ‘0’ to ‘1’ at the timing t4. The OUT_STOP signal output from the arithmetic unit 304 changes the OUT_VALID signal from ‘1’ to ‘0’ via the logic gate 124. Further, since the VALID_OUT signal of the arithmetic circuit 308 is “1”, the STOP_OUT signal is changed from “0” to “1” via the logic gate 122 so that d4 of DATA_OUT is not updated at the next timing t13. .
[0080]
At timing t13, the OUT_STOP signal output from the arithmetic unit 304 is “1”, the VALID_OUT signal output from the arithmetic circuit 308 is “1”, and the IN_VALID signal input from the arithmetic unit 300 is also “1”. . This is a situation where the preceding arithmetic unit 300 and the arithmetic unit 302 are trying to transfer valid data at a timing when the subsequent arithmetic unit 304 cannot receive data.
[0081]
The effective DATA_OUT d4 output from the arithmetic circuit 308 is not updated at the timing t13 because the STOP_OUT signal is set to '1' as described at the timing t12. Similarly to the timing t4, the VALID_IN signal and the STOP_OUT signal input to the arithmetic circuit 104 set the D-FF 112 to '1' via the logic gate 118 and the IN_STOP signal output to the arithmetic unit 300. Set to '1' and have the selector 114 select the buffer 110 side. The IN_STOP signal changes the IN_VALID signal from ‘1’ to ‘0’ by the same operation as the logic gate 124 described above. Further, D8 of IN_DATA is read into the buffer 110, and the read data D8 * is connected to DATA_IN of the arithmetic circuit 308.
[0082]
At timing t14, since the IN_STOP signal input to the arithmetic unit 300 is “1”, IN_DATA is not updated. Since the OUT_STOP signal input to the arithmetic unit 304 remains “1”, the DATA_OUT signal output from the arithmetic circuit 308 is not updated either. Similar to the processing of the IN_STOP signal output to the arithmetic unit 300 at timing t6, the OUT_STOP signal changes from ‘1’ to ‘0’ as a result of the processing of the arithmetic unit 304 at timing t14. The OUT_VALID signal output from the arithmetic unit 100 via the logic gate 124 is changed to ‘1’, and the STOP_OUT signal input to the arithmetic circuit 308 via the logic gate 122 is changed to ‘0’.
[0083]
At timing t15, since the OUT_VALID signal input to the arithmetic unit 304 is “1” and the OUT_STOP signal output from the arithmetic unit 304 is “0”, d4 of DATA_OUT output from the arithmetic circuit 308 is valid data. Are processed in the arithmetic unit 304, and DATA_OUT is updated. Further, since the IN_STOP signal input to the arithmetic unit 300 is “1”, IN_DATA output from the arithmetic unit 300 is not updated. However, since the VALID_IN signal input to the arithmetic circuit 308 is '1', the STOP_IN signal output from the arithmetic circuit 308 and the STOP_OUT signal input to the arithmetic circuit 308 are '0', the buffer connected to DATA_IN 110 data D8 * is input to the arithmetic circuit 308 and processed. The output of the logic gate 122 resets the D-FF 112 to “0” via the logic gate 118 and sets the IN_STOP signal output to the arithmetic unit 300 to “0”. At substantially the same time, the selector 114 selects the data input side. Further, the IN_STOP signal changes the IN_VALID signal from ‘0’ to ‘1’.
[0084]
At timing t16, similarly to timing t7, D9 of IN_DATA output from the arithmetic unit 300 is processed by the arithmetic circuit 308, and IN_DATA is updated.
[0085]
At timing t17, the OUT_STOP signal output from the arithmetic unit 304 is ‘1’, but there is no problem because neither the arithmetic unit 300 nor the arithmetic unit 302 attempts to transfer valid data.
[0086]
As described above, the function of the interface circuit 102 is that the operation unit 302 stops operating when the operation unit 302 corresponding to the data conversion circuit 100 cannot receive data or when the operation unit 304 at the subsequent stage cannot receive data. In such a case, the data transferred from the preceding arithmetic unit 300 is temporarily stored in the buffer, and the data updating of the preceding arithmetic unit 300 is stopped. Further, the interface circuit 102 can be used as a template for all the arithmetic units (circuit blocks) only by changing a part of the configuration as shown below.
[0087]
FIG. 9 and FIG. 10 are block diagrams respectively showing operations at the time of compression and expansion of the interface circuit 102, and the basic configuration is the same as FIG.
[0088]
At the time of compression, there are two types of data output from the data conversion core 104, and there are two subsequent blocks, the JPEG circuit 80 and the memory control circuit 40b. Therefore, the OR gate 120 is changed to three inputs to support two types of output data.
[0089]
At the time of decompression, there are two types of data to be input to the data conversion core 104, and there are two blocks in the previous stage: the JPEG circuit 80 and the memory control circuit 40d. Therefore, the STOP signal output from the memory control circuit 40c corresponding to the subsequent block is supplied to the OR gates 120a and 120b to correspond to the two types of input data.
[0090]
[Image recording process]
When the shooting mode is set in the mode dial 32, the image processing apparatus 1000 performs image processing on the CCD-RAW data of the shot image obtained by the shooting lens 10, the image sensor 12, and the A / D converter 14, or It is stored directly in the recording medium 70. FIG. 11 is a flowchart showing an example of the operation of the image processing apparatus 1000 relating to the recording processing of the photographed image, which is executed by the system control circuit 30.
[0091]
In FIG. 11, one of the Huffman tables stored in the read-only memory 36 is set in the JPEG circuit 80 (S100). The Huffman table used differs depending on whether the recording mode is JPEG mode or CCD-RAW lossless compression mode, and is suitable for Huffman tables suitable for DCT data sequences or DPCM converted data sequences. A Huffman table is used.
[0092]
The read-only memory 36 stores a plurality of Huffman tables for each of the two types of data series. These Huffman tables are generated from average histograms obtained from a plurality of image data each having highly similar features. Therefore, efficient encoding is provided for image data similar to the original image data.
[0093]
Next, the setting of the recording mode switch 34 is determined (S101). If the recording mode is the JPEG mode, the CCD-RAW data held in the memory 60 is sent to the image processing circuit 50 via the memory control circuit 40 (S102). The image processing circuit 50 performs image processing on the CCD-RAW data (S103). The image-processed data is sent to the JPEG circuit 80 via the memory control circuit 40, and is subjected to DCT and quantization (S104).
[0094]
On the other hand, if the setting of the recording mode switch 34 is the CCD-RAW mode, the CCD-RAW data held in the memory 60 is sent to the data conversion core 104 via the memory control circuit 40 and the interface circuit 102 (S110 ). Then, the data width of the CCD-RAW data is determined (S111), and in the case of 12 bits, the division / combination circuit 106 performs division / PACK processing (S113), and the packed data is sent via the memory control circuit 40. The data is sent to the memory 60 (S114). The 10-bit wide CCD-RAW data or the upper 10-bit data separated from the 12-bit wide CCD-RAW data is subjected to DPCM conversion by the DPCM conversion circuit 108 (S112).
[0095]
The DCT and quantized or DPCM-converted data is Huffman encoded according to the Huffman table set in the Huffman encoding / decoding circuit 86 of the JPEG circuit 80 (S105). The Huffman encoded data is sent to the memory 60 via the memory control circuit 40 (S106).
[0096]
For example, when the code for one image is stored in the memory 60, the data size of the Huffman code newly stored in the memory 60 and the Huffman code stored in the memory 60 by the previous encoding are compared (S107). The Huffman code having a small data size is held in the memory 60. Subsequently, it is determined whether or not all the Huffman tables have been encoded (S108) .If not completed, the process returns to Step 100, a new Huffman table is set in the JPEG circuit 80, and the encoding is repeated. . By repeating this encoding, the one having the smallest data size is selected from a plurality of Huffman codes obtained by different Huffman tables.
[0097]
Thereafter, the code stored in the memory 60 is stored as a data file in the recording medium 70 via the memory control circuit 40 (S109). At that time, the Huffman encoded data and the PACKed data are stored as CCD-RAW lossless compression data together with header data including a recording (photographing) condition. The header data will be described later with reference to FIG.
[0098]
As described above, in the present embodiment, a plurality of Huffman tables are prepared in advance, and encoding using a single Huffman table provided as a standard is performed by performing encoding on a single data multiple times. Compared to the above, it is possible to perform compression more effective in terms of compression rate.
[0099]
● Playback mode
When the playback mode is set in the mode dial 32, the image processing apparatus 1000 displays the image recorded in the image data file stored in the recording medium 70 on the image display unit 22. FIG. 12 is a flowchart showing an example of the operation of the image processing apparatus 1000 relating to image reproduction processing, which is executed by the system control circuit 30.
[0100]
The image data file stored in the recording medium 70 is read into the memory 60 via the memory control circuit 40 (S201), and the header data is analyzed by the system control circuit 30 (S202). The result of analyzing the header data is determined (S203). If the recording mode is the JPEG mode, the JPEG data held in the memory 60 is sent to the JPEG circuit 80 via the memory control circuit 40, and the JPEG data is decoded. (S204). That is, the JPEG data is Huffman-decoded by the Huffman code decoding circuit 86, and dequantized and inverse DCTed by the DCT / quantization circuit 82. Thereafter, the decoded image data is stored in the memory 60 via the memory control circuit 40 (S205). The image data held in the memory 60 is sent to the image display unit 22 via the memory control circuit 40 and the D / A converter 20, and an image is displayed (S206).
[0101]
If the result of analyzing the header data is the CCD-RAW mode, the Huffman-encoded data held in the memory 60 is sent to the JPEG circuit 80 via the memory control circuit 40 and sent to the Huffman encoding / decoding circuit 86. Decoding is performed (S207), and the Huffman-decoded data is subjected to inverse DPCM conversion by the DPCM conversion circuit 108 (S208).
[0102]
Then, the analysis result of the header data is determined (S209). When the CCD-RAW data is 10 bits, the 10-bit CCD-RAW data obtained by the inverse DPCM conversion is converted to the interface circuit 102 and the memory control circuit. It is stored in the memory 60 via 40 (S211). If the CCD-RAW data is 12 bits, the PACK data held in the memory 60 is sent to the data conversion core 104 via the memory control circuit 40 and the interface circuit 102, and is sent to the dividing / synthesizing circuit 106 by UNPACK. Processing and synthesis processing of lower 2 bits data obtained by UNPACK processing and upper 10 bits data obtained by inverse DPCM conversion are performed (S210). The restored 12-bit CCD-RAW data is stored in the memory 60 via the interface circuit 102 and the memory control circuit 40 (S211).
[0103]
The CCD-RAW data stored in the memory 60 is sent to the image processing circuit 50 via the memory control circuit 40 and subjected to image processing (S212). The RGB image data obtained by the image processing is stored in the memory 60 via the image processing circuit 40 (S205). The image data held in the memory 60 is sent to the image display unit 22 via the memory control circuit 40 and the D / A converter 20, and an image is displayed (S206).
[0104]
● Data format
FIG. 13 is a diagram showing a basic format example of CCD-RAW lossless compression data recorded on the recording medium 70. As shown in FIG.
[0105]
Header data is recorded at the head of the basic format shown in FIG. The first 1 bit of the header data indicates the recording mode MODE. When it is “0”, it indicates 10-bit CCD-RAW data, and when it is “1”, it indicates 12-bit CCD-RAW data. In the subsequent 15 bits, a 12-bit numerical value SIZE_H representing the number of pixels in the horizontal direction of the image sensor 12 is recorded in a bottom-up manner. In the subsequent 16 bits, a 12-bit numerical value SIZE_V indicating the number of pixels in the vertical direction of the image sensor 12 is recorded in a bottom-up manner. Subsequently, PACK data having a data size calculated from the following equation is recorded, and a Huffman code is recorded following the PACK data.
Data size = MODE x SIZE_H x SIZE_V / (8 x 8) [bytes]
[0106]
Also, whether the data file is a CCD-RAW lossless compression data file or a JPEG file can be determined by the first 4 bits of the data file. In the case of a JPEG file, since the file starts with the code “FFD8”, the first 4 bits are “1111”. On the other hand, in the case of CCD-RAW lossless compression data, it becomes '0000' or '1000'.
[0107]
As described above, according to the present embodiment, a CCD-RAW lossless compression / decompression circuit for 10- or 12-bit CCD-RAW data is realized using an existing baseline JPEG encoding / decoding circuit. Can do. That is, the JPEG recording mode or the CCD-RAW lossless compression recording mode can be selected according to the purpose of use of the image data. Accordingly, when it is desired to prevent data loss (image quality degradation) due to quantization when JPEG encoding is performed, CCD-RAW data can be reversibly compressed and stored in a recording medium. In addition, the compressed CCD-RAW data (especially 12-bit CCD-RAW data) can have a data size that is sufficiently smaller than when data is stored in the CCD-RAW format.
[0108]
In other words, a part of the existing baseline JPEG encoding / decoding circuit can be used for CCD-RAW lossless compression / decompression processing, so the load on development of equipment that requires such a circuit. Such a device can be realized with a smaller circuit scale.
[0109]
Furthermore, in the present embodiment, by preparing a plurality of Huffman tables in advance and performing encoding a plurality of times on one data, rather than compression by a single standard Huffman table provided, Effective compression can be performed in terms of compression rate.
[0110]
Since the selection and supply of the Huffman table is selectively performed by the system control unit 30, it is easily realized without changing the image compression / decompression unit. Furthermore, since it is not necessary to generate a Huffman table for each data to be compressed, it is possible to reduce a load such as software development for generating the Huffman table.
[0111]
In the above-described embodiment, an image processing apparatus including an image sensor such as an electronic camera (digital camera) has been described. However, similar processing can be performed by a computer, an image output apparatus, or the like.
[0112]
[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.
[0113]
Also, an object of the present invention is to supply a storage medium (or recording medium) on which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and the computer (or CPU or CPU) of the system or apparatus Needless to say, this can also be achieved by the MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0114]
Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
[0115]
When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above (shown in FIG. 11 and / or FIG. 12).
[0116]
The application of the present invention is not limited to an image processing apparatus such as a digital camera and its recording medium. For example, when distributing image data of an image taken by an image processing device such as a digital camera using a medium such as a CD-ROM or DVD-ROM, for normal use such as display, printing, and search. The image data to be used is JPEG encoded and recorded on a CD-ROM, etc. The image data that requires any image processing such as color separation for printing, CG creation, etc. is in the CCD-RAW format on a CD-ROM etc. When recording, the image data compression / decompression method of the present invention can be used. Further, the present invention includes media in which image data is recorded (stored) in such a form.
[0117]
【The invention's effect】
  As explained above, according to the present invention,Image data (for example, CCD-RAW Data)An image processing apparatus that realizes effective encoding can be provided.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration example of an image processing apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram showing a data format in division and PACK processing, and UNPACK and synthesis processing;
FIG. 3 is a diagram showing an example of a color filter array;
FIG. 4 is a block diagram showing a detailed configuration example of the data conversion core shown in FIG.
FIG. 5 is a diagram showing an example of the data format of CCD-RAW data;
FIG. 6 is a timing chart showing a specific example of waiting processing;
7 is a block diagram showing a most basic configuration example of the interface circuit shown in FIG.
FIG. 8A is a timing chart showing in detail the relationship between data, a VALID signal, and a STOP signal;
FIG. 8B is a timing chart showing in detail the relationship between data, a VALID signal, and a STOP signal;
FIG. 9 is a block diagram showing the compression operation of the interface circuit;
FIG. 10 is a block diagram showing an expansion operation of the interface circuit;
FIG. 11 is a flowchart illustrating an operation example of the image processing apparatus related to a recording process of a captured image;
FIG. 12 is a flowchart showing an operation example of the image processing apparatus related to image reproduction processing;
FIG. 13 is a diagram illustrating a basic format example of CCD-RAW lossless compression data recorded on a recording medium.

Claims (13)

Conversion means for digitizing the output signal of the image sensor and converting it to first image data;
When the number of bits of the first image data is the first number of bits, the first image data is encoded according to a first encoding method, and the number of bits of the first image data is the first number of bits. In the case where the second bit number is larger than the bit number of 1, the first image data is divided into the first partial data for the first bit number and a part other than the first partial data. Encoding means for dividing the first partial data according to the first encoding scheme, and dividing the first partial data into second partial data having
In the case where the first image data is stored in a recording medium, and the number of bits of the first image data is the first number of bits, the encoding unit encodes the first image data. The first image data is stored in the recording medium, the first image data is stored in the recording medium, and the number of bits of the first image data is the second number of bits. In some cases, the image processing apparatus includes recording means for storing the first partial data encoded by the encoding means and the second partial data in the recording medium.   The encoding means is an encoding means for encoding the first image data or the first partial data using a Huffman table having a minimum data size selected from a plurality of Huffman tables. 2. The image processing apparatus according to claim 1, wherein:   The encoding means is an encoding means for encoding the first partial data according to the first encoding method, but not encoding the second partial data according to the first encoding method. The image processing apparatus according to claim 1, wherein: Image processing means for performing image processing on the first image data and generating second image data;
The encoding means is an encoding means for encoding the second image data according to a second encoding scheme different from the first encoding scheme;
The recording unit stores the second image data encoded by the second encoding unit in the recording medium when storing the second image data in the recording medium. the image processing apparatus according to any one of claims 1 to 3,.   5. The image processing apparatus according to claim 4, wherein the first encoding method and the second encoding method are encoding methods including Huffman encoding.   The encoding means is an encoding means for encoding the second image data using a Huffman table having a minimum data size selected from a plurality of Huffman tables. The image processing apparatus according to 4 or 5. The second encoding method is a discrete cosine transform and the image processing apparatus according to any one of claims 4 to 6, characterized in that the encoding method including quantization. The image processing, pixel interpolation processing, the image processing apparatus according to any one of claims 4, wherein 7 in that it comprises at least one color conversion process. The second image data, the image processing apparatus according to any one of claims 4, characterized in that the RGB image data 8. Wherein the first coding method, an image processing apparatus according to any one of claims 1 9, characterized in that a coding scheme for performing predictive coding. Wherein the first coding method, an image processing apparatus according to the difference value of two pixels from claim 1, characterized in that the coding method for Huffman coding in any one of 10. The first image data, the image processing apparatus according to any one of claims 1 to 11, characterized in that a CCD-RAW data. The recording medium is, the image processing device according to claim 1, wherein in any one of 12 to be a removable recording medium.

Images