JP2010147531A - Imaging apparatus and imaging method - Google Patents

Abstract

An imaging apparatus and an imaging method capable of high-speed processing are provided.
A variable-length RAW compression unit 22 performs variable-length compression processing of RAW data, and temporarily stores the generated variable-length compressed data in an SDRAM 41. The variable length RAW decompression unit 23 decompresses stored variable length compressed data to generate RAW data. This RAW data is subjected to fixed-length compression processing by the fixed-length RAW compression unit 24, and the generated fixed-length compressed data is temporarily stored in the SDRAM 41. The fixed-length RAW decompression unit 25 performs decompression processing on the stored fixed-length compressed data for each area obtained by dividing the entire screen in the line direction, and performs development processing on the RAW data generated by performing the decompression processing. This is performed by the signal processing unit 26. Since RAW data is compressed by a variable length compression method, the amount of data is reduced and high-speed processing is possible.
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus and an imaging method. Specifically, by performing compression processing and decompression processing of the variable length method and the fixed length method, it is possible to efficiently perform temporary storage and development processing of image data obtained by digitally converting the output from the image sensor, for example, RAW data. To.

  In recent years, in imaging apparatuses such as digital still cameras and digital video cameras, the number of pixels in an imaging element has increased and the functions and performance of the apparatus have been improved. In particular, as the number of pixels of the image sensor increases, the processing load of the image pickup signal increases. However, even such an image pickup apparatus is required to be able to process at high speed so that there is no stress in operation. Therefore, in Patent Document 1, as shown in FIG. 1, the RAW data generated by the AFE unit is compressed by the RAW compression unit and held in the SDRAM, and the compressed data read from the SDRAM is expanded by the RAW expansion unit. Thus, it is disclosed that RAW development processing is performed in the RAW development processing unit.

JP 2007-228515 A

  By the way, when decompressing compressed data and performing RAW development processing, etc., the entire screen is divided in the line direction and processed for each region, so that the line memory, delay line, etc. required for RAW development processing, etc. Can reduce the scale. However, in the processing for each area, it is necessary to perform random access to the SDRAM and read out necessary data. Thus, in order to be able to read data by random access, a fixed-length compression method in which the data amount is constant must be used as the compression method, and the data amount of the compressed data cannot be reduced sufficiently. . Also, since the amount of compressed data cannot be reduced sufficiently, high-speed processing is difficult.

  Accordingly, an object of the present invention is to provide an imaging device and an imaging method that enable high-speed processing.

  According to a first aspect of the present invention, there is provided a variable length compression unit that performs compression processing by a variable length compression method on image data obtained by digitally converting an output from an image sensor, and generates variable length compressed data; A variable-length compressed data storage unit that temporarily stores variable-length compressed data, a variable-length decompression unit that performs decompression processing of the stored variable-length compressed data, and an image generated by performing decompression processing using the variable-length decompression unit A fixed-length compression unit that compresses data using a fixed-length compression method and generates fixed-length compressed data; a fixed-length compressed data storage unit that temporarily stores fixed-length compressed data; and the entire screen in the line direction A fixed-length decompression unit that performs segmentation and performs decompression processing of the fixed-length compressed data corresponding to the segment for each segmented region, and image data generated by performing decompression processing in the fixed-length decompression unit , Ward In an imaging device having a developing unit for performing been for each region the development process.

  In the present invention, variable-length compressed data generated by compressing raw data generated by single-shot imaging or continuous-shot imaging using a variable-length compression method is stored in a variable-length compressed data storage unit. . The stored variable-length compressed data is decompressed, compressed by a fixed-length compression method, and stored in a fixed-length compressed data storage unit. The stored fixed-length compressed data is expanded for each area provided in a strip shape, for example, by dividing the entire screen in the line direction, and the image data generated by performing the expansion processing is developed. The

  The variable length compression unit, the variable length compressed data storage unit, the variable length expansion unit, and the fixed length compression unit, the fixed length compression data storage unit, and the fixed length expansion unit are connected via a bus. In addition, a fixed length compression data storage unit connected to the fixed length compression unit and the fixed length decompression unit without using the bus is provided, and the fixed data generated by compressing the RAW data using the fixed length compression method. The long compressed data is stored in the second fixed length compressed data storage unit. Here, when the variable-length compressed data exceeds a predetermined amount of data, the fixed-length compressed data stored in the second fixed-length compressed data storage unit is decompressed for each region and generated by performing decompression processing. The processed image data is developed.

  According to a second aspect of the present invention, the variable length compression unit performs compression processing of the variable length compression method on the image data obtained by digitally converting the output from the image sensor, and generates variable length compressed data. A step of temporarily storing the variable length compressed data in the variable length compressed data storage unit, a step of performing a decompression process of the stored variable length compressed data in the variable length decompression unit, and a decompression process in the variable length decompression unit The fixed length compression unit compresses the image data generated by performing the fixed length compression method to generate fixed length compressed data, and the fixed length compressed data storage unit A step of temporarily storing the entire screen in a line direction, and a step of performing a decompression process of the fixed-length compressed data corresponding to the divided area for each of the divided areas by the fixed-length decompression unit , By using the image data generated by performing the decompression process in a fixed-length decompression unit, in the imaging device and performing a development process for each sectioned area.

  According to the present invention, the variable length compression method compression processing is performed on the image data obtained by digitally converting the output from the image sensor, and the variable length compression data generated by the compression processing is variable. Temporarily stored in the long compressed data storage unit. Also, the fixed-length compression data generated by the compression processing is performed on the image data generated by performing the expansion processing of the stored variable-length compression data, and the compression processing of the fixed-length compression method is performed. Temporarily stored in the fixed-length compressed data storage unit. Further, the stored fixed-length compressed data is expanded for each area obtained by dividing the entire screen in the line direction, and image data generated by performing this expansion processing is developed.

  For this reason, for example, RAW data generated by performing single-shot imaging or continuous shooting imaging is temporarily stored as variable-length compressed data, so that a high-speed imaging operation can be performed. The temporarily stored fixed-length compressed data is expanded for each area obtained by dividing the entire screen in the line direction, and the image data generated by performing the expansion processing is developed. Therefore, the development processing can be performed at high speed by reducing the scale of the line memory, the delay line, and the like.

The best mode for carrying out the invention will be described below. The description will be given in the following order.
1. Configuration of the first embodiment 2. Operation of the first embodiment Configuration of second embodiment 4. Operation of the second embodiment

<1. Configuration of First Embodiment>
FIG. 2 is a block diagram showing the configuration of the first embodiment. The imaging apparatus 10 includes an imaging optical system block 11, an imaging unit 12, an analog front end (AFE) unit 13, and a signal processing unit 20. Furthermore, the imaging apparatus 10 includes an SDRAM 41, a ROM 42, a recording / playback unit 43, a display unit 44, and an operation unit 45.

  The signal processing unit 20 includes a camera signal preprocessing unit 21, a variable length RAW compression unit 22, a variable length RAW expansion unit 23, a fixed length RAW compression unit 24, a fixed length RAW expansion unit 25, and a camera signal processing unit 26. ing. Further, the signal processing unit 20 includes a resolution conversion unit 27, a JPEG (Joint Photographic Experts Group) engine 28, a CPU 29, a video output encoder 30, and an SDRAM controller 31. In the signal processing unit 20, the blocks are connected to each other via a bus 32.

  The imaging optical system block 11 includes a zoom lens that performs zooming, a focus lens that performs focusing, a diaphragm mechanism that adjusts the amount of light, and a drive unit that drives the zoom lens, the focus lens, and the diaphragm mechanism.

  The imaging unit 12 includes, for example, an imaging element such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS) type image sensor and a driving unit that drives the imaging element. The imaging unit 12 converts the subject image formed on the imaging surface of the imaging element by the imaging optical system block 11 into an electrical signal and outputs the electrical signal to the AFE unit 13.

  The AFE unit 13 performs noise removal processing such as CDS (Correlated Double Sampling) processing on the electrical signal (imaging signal) output from the imaging unit 12, and AGC (Automatic Gain Control) using the imaging signal as a desired signal level. Process. Further, the AFE unit 13 converts an analog imaging signal subjected to noise removal processing and gain control into a digital signal and outputs the digital signal to the signal processing unit 20.

  The signal processing unit 20 is formed as a SoC (System On a Chip) circuit, for example. The camera signal preprocessing unit 21 of the signal processing unit 20 performs a defect correction process for correcting a signal of a defective pixel in the image sensor on the image signal supplied from the AFE unit 13, and a shading for correcting a peripheral light amount drop of the lens. Correction processing and the like are performed and output as RAW data.

  The variable length RAW compression unit 22 compresses the RAW data supplied from the camera signal preprocessing unit 21 using a variable length encoding method. The variable length RAW decompression unit 23 compresses the RAW data by, for example, a reversible compression method using a Huffman table to generate variable length compressed data. Further, the variable length RAW compression unit 22 stores the variable length compressed data generated by performing the compression process in the SDRAM 41 via the bus 32 or the SDRAM controller 31.

  The variable length RAW decompression unit 23 decompresses variable length compressed data read from the SDRAM 41 and supplied via the SDRAM controller 31 or the bus 32. In addition, the variable length RAW decompression unit 23 outputs the RAW data generated by performing the decompression process to the fixed length RAW compression unit 24.

  The fixed-length RAW compression unit 24 compresses the RAW data supplied from the variable-length RAW expansion unit 23 using a fixed-length encoding method. The fixed-length RAW compression unit 24 generates fixed-length compressed data with a constant data amount, for example, with a quantized word length per pixel as a fixed length. The fixed-length RAW compression unit 24 stores the fixed-length compressed data generated by performing the compression process in the SDRAM 41 via the bus 32 or the SDRAM controller 31.

  The fixed length RAW decompression unit 25 decompresses fixed length compressed data read from the SDRAM 41 and supplied via the SDRAM controller 31 or the bus 32. The fixed-length RAW expansion unit 25 outputs the RAW data generated by performing the expansion process to the camera signal processing unit 26.

  The camera signal processing unit 26 performs RAW development processing of RAW data supplied from the fixed-length RAW decompression unit 25, detection processing for AF (Auto Focus), AE (Auto Exposure), or a part of these processing. Execute.

  In the RAW development process, a demosaic process is performed. In order to display a color image, for example, red, green, and blue values are required for each pixel. However, when one imaging device is used in the imaging unit 12 and a color filter array is provided in front of the imaging surface, one pixel is, for example, one of red, green, and blue Only the signal value of is obtained. Therefore, demosaic processing is performed to generate signals of the other two missing colors by interpolation or the like using signal values of surrounding pixels. The signal values of the surrounding pixels can be obtained from the previous line or the next line by using, for example, a line memory or a delay line.

  In the RAW development processing, correction processing such as brightness correction, color correction, and white balance adjustment is performed. Furthermore, a desired format, for example, a process of converting to luminance data (Y) and color difference data (RY, BY) in which the ratio of luminance information and color difference information is “4: 2: 2” is performed.

  The resolution conversion unit 27 converts the image data output from the camera signal processing unit 26 or the image data decompressed and decoded by the JPEG engine 28 into image data having a predetermined resolution.

  The JPEG engine 28 compresses and encodes the image data output from the resolution conversion unit 27, and generates JPEG encoded data. The JPEG engine 28 decompresses and decodes the JPEG image data read from the recording / playback unit 43. Note that the signal processing unit 20 may be provided with an encoding / decoding engine of a still image compression method or a moving image compression method other than the JPEG engine 28.

  The CPU 29 executes a program stored in the ROM 42 to control the signal processing unit 20 and the entire imaging apparatus as a whole, and executes various calculations for the control.

  The video output encoder 30 converts the output image data supplied from the resolution converter 27 and the JPEG engine 28 into a format corresponding to a device connected to the video output terminal 30a, and converts the converted image data to the video output terminal. Output from 30a. Further, the video output encoder 30 converts the output image data into display data for monitor display and outputs the display data to the display unit 44 described later.

  The SDRAM controller 31 is an interface to the SDRAM 41 and includes an address decoder. The SDRAM controller 31 controls a data write operation to the SDRAM 41 and a data read operation from the SDRAM 41 in accordance with a control signal from the CPU 29.

  The SDRAM 41 is a volatile memory used as a work area for data processing or the like in the signal processing unit 20. The SDRAM 41 is provided with a capture data area 411, a JPEG code area 412, a CPU work area 413, and the like. The capture data area 411 is an area for temporarily storing variable-length compressed data generated by the variable-length RAW compressing unit 22 and fixed-length compressed data generated by the fixed te fixed-length RAW compressing unit 24. The JPEG code area 412 is an area for temporarily storing image data encoded by the JPEG engine 28, data used in the encoding / decoding process, and the like. The CPU work area 413 is an area for temporarily storing data used for processing by the CPU 29.

  The ROM 42 stores programs executed by the CPU 29 and various data. As the ROM 42, for example, a nonvolatile memory such as an EPROM (Electronically Erasable and Programmable ROM) flash memory is used.

  In the recording / reproducing unit 43, for example, a recording medium such as a flash memory, an optical disk, or a magnetic tape is used. The recording / reproducing unit 43 records the image data output from the signal processing unit 20 on a recording medium. Further, the recording / reproducing unit 43 performs processing of reading out image data recorded on the recording medium and supplying the read image data to the signal processing unit 20. Note that the recording medium used in the recording / reproducing unit 43 may be detachable or may not be detachable.

  The display unit 44 displays a camera-through image captured by the imaging apparatus 10 and a captured image recorded in the recording / playback unit 43. The display unit 44 also displays a menu for setting the imaging device 10 and the like.

  The operation unit 45 includes operation buttons and a touch panel provided on the screen of the display unit 44. The operation unit 45 generates an operation signal corresponding to a user operation such as a shutter operation and outputs the operation signal to the CPU 29 of the signal processing unit 20.

<2. Operation of First Embodiment>
When the shutter operation is performed by the operation unit 45, the imaging device 10 performs the processing of the flowchart illustrated in FIG. In FIG. 3, the CPU 29 of the signal processing unit 20 performs an imaging operation in step ST1. The CPU 29 controls the imaging unit 12 to perform imaging with a desired exposure time, stores variable-length compressed data of the captured image in the capture data area 411 of the SDRAM 41, and proceeds to step ST2.

  FIG. 4 shows a signal path when an imaging operation is performed. The imaging signal generated by the imaging unit 12 by performing imaging with a desired exposure time is subjected to noise removal processing or the like by the AFE unit 13, and then converted into a digital signal to be preprocessed by the signal processing unit 20. Supplied to the unit 21.

  The camera signal preprocessing unit 21 performs defect correction processing, shading correction processing, and the like, and outputs the processed signal to the variable length RAW compression unit 22 as RAW data.

  The variable length RAW compression unit 22 compresses the RAW data supplied from the camera signal preprocessing unit 21 using a variable length encoding method. Further, the variable length RAW compression unit 22 stores the variable length compressed data generated by performing the compression processing in the capture data area 411 of the SDRAM 41 via the bus 32 or the SDRAM controller 31.

  By performing the processing in this manner, a captured image generated by performing imaging with a desired exposure time can be stored in the capture data area 411 of the SDRAM 41.

  In step ST <b> 2, the CPU 29 determines whether or not the imaging is finished. When the shooting mode is set to the continuous shooting mode, the CPU 29 determines whether or not variable-length compressed data for a preset number of continuous shots is stored in the capture data area 411 of the SDRAM 41. The CPU 29 returns to step ST1 when the variable-length compressed data for the number of continuous shots is not stored in the capture data area 411 of the SDRAM 41. In addition, when the variable-length compressed data for the number of continuous shots is stored in the capture data area 411 of the SDRAM 41, the CPU 29 determines that the imaging operation is finished and proceeds to step ST3.

  When the image capturing mode is set to the single image capturing mode, the variable length compressed data of the captured image when the shutter operation is performed is stored in the capture data area 411 of the SDRAM 41 by performing the process of step ST1. . Therefore, since the variable length compressed data for one sheet is stored in the capture data area 411 of the SDRAM 41, the CPU 29 determines that the imaging operation is finished and proceeds to step ST3.

  In step ST3, the CPU 29 performs compression rate conversion. The CPU 29 controls the variable-length RAW decompression unit 23, the fixed-length RAW compression unit 24, and the SDRAM controller 31 to convert one piece of variable-length compressed data stored in the capture data area 411 of the SDRAM 41 into fixed-length compressed data. The conversion proceeds to step ST4.

  FIG. 5 shows a signal path when the compression rate conversion operation is performed. The CPU 29 supplies the variable-length compressed data for one sheet stored in the capture data area 411 of the SDRAM 41 to the variable-length RAW decompression unit 23 via the SDRAM controller 31 and the bus 32.

  The variable length RAW decompression unit 23 decompresses variable length compressed data read from the capture data area 411 of the SDRAM 41. Further, the variable length RAW decompression unit 23 outputs the RAW data obtained by performing the decompression process to the fixed length RAW compression unit 24.

  The fixed-length RAW compression unit 24 compresses the RAW data supplied from the variable-length RAW expansion unit 23 using a fixed-length encoding method. The fixed-length RAW compression unit 24 stores the fixed-length compressed data obtained by performing the compression process in the capture data area 411 of the SDRAM 41 via the bus 32 or the SDRAM controller 31.

  As described above, when the compression rate conversion operation is performed, one fixed-length compressed data can be stored in the capture data area 411 of the SDRAM 41.

  In step ST4, the CPU 29 performs RAW development processing. The CPU 29 controls the fixed length RAW expansion unit 25 and the SDRAM controller 31 to read and expand the fixed length compressed data stored in the capture data area 411 of the SDRAM 41. Further, the CPU 29 controls the camera signal processing unit 26 to perform RAW development processing of the RAW data generated by the fixed length RAW expansion unit 25.

  Here, the camera signal processing unit 26 uses a delay line that is about a fraction of one horizontal period in order to suppress the scale of the delay line, for example. In this case, the CPU 29 divides the entire screen in the line direction according to the delay line, that is, divides the entire screen into a plurality of vertical strips as shown in FIG. Perform development processing.

  Also, fixed length compressed data is stored in the capture data area 411 of the SDRAM 41. Therefore, the CPU 29 can easily calculate the address where the fixed-length compressed data corresponding to the area is stored. Further, the CPU 29 reads out the fixed-length compressed data corresponding to the portion where the RAW development processing is performed from the capture data area 411 of the SDRAM 41 using the calculated address, and supplies it to the fixed-length RAW decompression unit 25.

  FIG. 7 shows a signal path when the development processing is performed. Based on the calculated address, the CPU 29 reads the data of the portion to be developed from the fixed-length compressed data stored in the capture data area 411 of the SDRAM 41. The CPU 29 also supplies the read data to the fixed-length RAW expansion unit 25 via the SDRAM controller 31 and the bus 32.

  The fixed length RAW decompression unit 25 performs decompression processing on the fixed length compressed data read from the capture data area 411 of the SDRAM 41. The fixed-length RAW expansion unit 25 outputs RAW data obtained by performing the expansion process to the camera signal processing unit 26.

  The camera signal processing unit 26 performs RAW development processing using the RAW data supplied from the fixed-length RAW expansion unit 25. The camera signal processing unit 26 sequentially performs RAW development processing for the entire screen for each region shown in FIG. Note that when peripheral pixels are used in the RAW development process as in the demosaic process, there is no data of adjacent pixels at the boundary of the strip-shaped region. For this reason, when reading out the fixed-length compressed data in the next area, if the areas are overlapped as shown in FIG. 6, the pixels at the boundary can be processed by the RAW development process in the next area.

  Further, the CPU 29 causes the resolution conversion unit 27 to perform resolution conversion, the JPEG engine 28 to perform compression encoding processing, or the like on the image data after the RAW development processing in accordance with settings or the like registered in advance by the user. Proceed to ST5.

  The CPU 29 records the image data after the RAW development processing or the image data on which the resolution conversion or the compression encoding processing has been performed on the image data after the RAW development processing on the recording medium of the recording / playback unit 43.

  In step ST5, the CPU 29 determines whether or not processing for the number of captured images has been completed. When the imaging mode is set to the continuous shooting mode, the CPU 29 has performed the compression rate conversion operation and the RAW development processing on the variable-length compressed data for the number of continuous shots stored in the capture data area 411 of the SDRAM 41. Determine whether or not. The CPU 29 returns to step ST3 when the variable length compressed data for which the compression rate conversion operation and the RAW development processing are not completed remain. At this time, the CPU 29 reads out the new variable length compressed data that has not been subjected to the compression rate conversion operation and the RAW development processing, and causes the processing in step ST3 and step ST4 to continue. Further, the CPU 29 ends the processing when the compression rate conversion operation and the RAW development processing are performed for each of the variable length compressed data for the number of continuous shots.

  When the shooting mode is set to the single shooting mode, only one variable-length compressed data is stored in the capture data area 411 of the SDRAM 41. Therefore, when the processes of step ST3 and step ST4 are performed, the process ends because there is no variable length compressed data that has not been subjected to compression rate conversion and RAW development processing.

  The image data after the RAW development processing may be recorded as it is on the recording medium by the recording / reproducing unit 43, or the image data is processed by the resolution converting unit 27 and / or the JPEG engine 28 and then recorded on the recording medium. Also good. Alternatively, the image data after the RAW development processing or the image data may be processed by the resolution conversion unit 27 and then encoded by the JPEG engine 28 and stored in the JPEG code area 412 of the SDRAM 41.

  As described above, if the RAW data is compressed by the variable-length encoding method and the variable-length compressed data is temporarily stored in the memory, the amount of data temporarily stored in the memory is reduced as compared with the case where the fixed-length encoding method is used. it can. Therefore, high-speed processing is possible as compared with a conventional imaging apparatus using a fixed-length encoding method, and even if the number of pixels of the imaging element is increased, continuous shooting is performed and a plurality of captured images are temporarily stored in the memory. Even cases can be easily handled.

  For example, when the SDRAM 41 is shared and each block connected to the bus 32 has an architecture capable of accessing the SDRAM 41 as in the imaging device 10, the SDRAM 41 can be effectively used. However, since each block accesses the SDRAM 41 via the bus 32, it is important to secure the bandwidth of the bus 32.

  Here, when writing and reading RAW data to and from the SDRAM 41, since data for the entire screen flows on the bus 32, the bus bandwidth necessary for this transmission occupies most of the entire bus bandwidth at the time of imaging. . In particular, as the number of pixels of the image sensor increases and the capacity of RAW data increases, the load of data transfer increases and the time required for writing / reading to the SDRAM 41 also increases. Therefore, if it is intended to shorten the time required for processing, it is necessary to expand the bus band by increasing the transmission frequency, which increases the device cost. In addition, as the number of pixels of the image sensor increases, the memory capacity of the SDRAM 41 must be increased. However, since the RAW data is compressed by the variable length compression method and temporarily stored in the SDRAM 41 as variable length compressed data, the amount of data is reduced and high speed processing is possible without increasing the bus bandwidth.

  Further, in order to suppress the scale of the delay line in the camera signal processing unit 26, for example, a delay line that is about a fraction of one horizontal period is used, and the entire screen is divided in the line direction according to the delay line. Development processing is performed for each of the areas. In this case, it is necessary that at least the data of the entire screen is stored in the SDRAM 41 before the processing, and the data required for signal processing can be read out from the SDRAM 41 at random. Here, when the RAW data is compressed by the variable length compression method, the data required for signal processing cannot be read out from the SDRAM 41 at random. However, when performing RAW development processing, the variable-length compressed data is expanded by the variable-length RAW expansion unit 23 and then compressed by the fixed-length RAW compression unit 24 and stored in the SDRAM 41 as fixed-length compressed data. Therefore, the data required from the fixed-length compressed data is randomly read out, the decompression process is performed by the fixed-length RAW decompression unit 25, and the RAW data generated by the fixed-length RAW decompression unit 25 is used for each divided area. RAW development processing can be performed.

  Note that the image sensor is not limited to the frame readout method, and may be an interlace readout method. In this case, if a plurality of fields of data are temporarily stored in the SDRAM 41 and the temporarily stored data of the plurality of fields are read in units of frames, the processing of the RAW data is performed in the same manner as when the image sensor is a frame reading method. It can be carried out.

<3. Configuration of Second Embodiment>
By the way, in the first embodiment, the RAW data is compressed using the variable length coding method, but the variable length coding method has a data amount after compression depending on the compression algorithm and the content of the image to be compressed. Change. For this reason, when the RAW data is compressed using the variable length coding method, the data amount of the variable length compressed data fluctuates. Therefore, the variable length RAW compression unit 22 has a buffer memory on the output side. Therefore, the imaging apparatus stores variable length compressed data in the buffer memory, and transfers the variable length compressed data from the buffer memory to the SDRAM 41 for storage. In this way, the variable length compressed data can be stored in the SDRAM 41 even if the bandwidth available for communication of the variable length compressed data on the bus 32 changes. It is also possible to store the variable length compressed data in the SDRAM 41 at a constant speed regardless of the data amount of the variable length compressed data.

  However, the data amount of the variable length compressed data may not be smaller than the predetermined data amount depending on the combination of the compression algorithm and the image content. For this reason, when the amount of variable-length compressed data increases, an overflow may occur beyond the capacity of the buffer memory. In addition, if the amount of variable-length compressed data is large, it may take a long time to transfer the variable-length compressed data, and the operation of the imaging apparatus 10 may fail.

  In such a case, the data amount of the variable length compressed data is monitored, and when the prescribed data amount is exceeded, the variable length RAW compression operation is stopped or the variable length compressed data is not written to the SDRAM 41 but discarded. If an operation is provided, failure of the operation can be prevented. However, when such an exceptional operation occurs, data captured at that time is lost. Further, in order to prevent the failure of the operation, a compression algorithm that does not cause the failure of the operation regardless of the image contents is required.

  Therefore, as a second embodiment, an imaging apparatus capable of performing RAW development processing or the like without causing an operation failure even when the data amount of variable-length compressed data does not become smaller than a predetermined data amount. Will be described.

  FIG. 8 is a block diagram showing the configuration of the second embodiment. The imaging device 10a includes an imaging optical system block 11, an imaging unit 12, an analog front end (AFE) unit 13, and a signal processing unit 20a. Further, the imaging apparatus 10 includes an SDRAM 41, a ROM 42, a recording / reproducing unit 43, a display unit 44, and an operation unit 45.

  The signal processing unit 20a includes a camera signal preprocessing unit 21, a variable length RAW compression unit 22, a variable length RAW expansion unit 23, a fixed length RAW compression unit 24, a fixed length RAW expansion unit 25, and a camera signal processing unit 26. ing. Further, the signal processing unit 20 a includes a resolution conversion unit 27, a JPEG engine 28, a CPU 29 a, a video output encoder 30, and an SDRAM controller 31. The signal processing unit 20a has a configuration in which the blocks are connected to each other by a bus 32. Further, the SRAM 33 is connected to the signal processing unit 20 to the fixed length RAW compression unit 24 and the fixed length RAW expansion unit 25 without going through the bus 32.

  The imaging optical system block 11 includes a zoom lens that performs zooming, a focus lens that performs focusing, a diaphragm mechanism that adjusts the amount of light, and a drive unit that drives the zoom lens, the focus lens, and the diaphragm mechanism.

  The imaging unit 12 includes, for example, an imaging element such as a CCD or CMOS image sensor and a driving unit that drives the imaging element. The imaging unit 12 converts the subject image formed on the imaging surface of the imaging element by the imaging optical system block 11 into an electrical signal and outputs the electrical signal to the AFE unit 13.

  The AFE unit 13 performs noise removal processing, such as CDS processing, or AGC processing for setting the imaging signal to a desired signal level on the electrical signal (imaging signal) output from the imaging unit 12. Further, the AFE unit 13 converts an analog imaging signal subjected to noise removal processing and gain control into a digital signal and outputs the digital signal to the signal processing unit 20a.

  The signal processing unit 20a is formed as a SoC circuit, for example. The camera signal preprocessing unit 21 of the signal processing unit 20a performs a defect correction process for correcting a signal of a defective pixel in the image sensor on the image signal supplied from the AFE unit 13, and a shading for correcting a peripheral light amount drop of the lens. Correction processing and the like are performed and output as RAW data.

  The variable length RAW compression unit 22a compresses the RAW data supplied from the camera signal preprocessing unit 21 using a variable length encoding method. The variable length RAW decompression unit 23 compresses the RAW data by, for example, a reversible compression method using a Huffman table to generate variable length compressed data. The variable-length RAW compression unit 22 a stores variable-length compressed data generated by performing the compression process in the SDRAM 41 via the bus 32 or the SDRAM controller 31. Further, the variable length RAW compression unit 22a monitors the data amount of the variable length compressed data and outputs a determination signal indicating whether or not the data amount exceeds a predetermined data amount to the CPU 29a. For example, it is determined whether or not a buffer memory (for example, FIFO) provided on the output side of the variable length RAW compression unit 22 has overflowed, and a determination signal indicating the determination result is output to the CPU 29a.

  The variable length RAW decompression unit 23 decompresses variable length compressed data read from the SDRAM 41 and supplied via the SDRAM controller 31 or the bus 32. In addition, the variable length RAW decompression unit 23 outputs the RAW data generated by performing the decompression process to the fixed length RAW compression unit 24.

  The fixed-length RAW compression unit 24 compresses the RAW data supplied from the camera signal preprocessing unit 21 and the variable-length RAW expansion unit 23 using a fixed-length encoding method. The fixed-length RAW compression unit 24 generates fixed-length compressed data with a constant data amount, for example, with a quantized word length per pixel as a fixed length. The fixed-length RAW compression unit 24 stores the fixed-length compressed data generated by performing the compression process in the SDRAM 41 or the SRAM 33 via the bus 32 or the SDRAM controller 31.

  The fixed-length RAW decompression unit 25 performs decompression processing on the fixed-length compressed data read from the SDRAM 41 and supplied via the SDRAM controller 31 and the bus 32 and the fixed-length compressed data read from the SRAM 33. The fixed-length RAW expansion unit 25 outputs the RAW data generated by performing the expansion process to the camera signal processing unit 26.

  The camera signal processing unit 26 executes RAW development processing of RAW data supplied from the fixed-length RAW decompression unit 25, detection processing for AF, AE, or the like, or part of these processing. In the RAW development processing, correction processing such as demosaic processing, brightness correction, color correction, and white balance adjustment is performed. Furthermore, the process etc. which convert into the signal of a desired format are performed.

  The resolution conversion unit 27 converts the image data output from the camera signal processing unit 26 or the image data decompressed and decoded by the JPEG engine 28 into image data having a predetermined resolution.

  The JPEG engine 28 compresses and encodes the image data output from the resolution conversion unit 27, and generates JPEG encoded data. The JPEG engine 28 decompresses and decodes the JPEG image data read from the recording / playback unit 43. Note that the signal processing unit 20 may be provided with an encoding / decoding engine of a still image compression method or a moving image compression method other than the JPEG engine 28.

  The CPU 29a executes a program stored in the ROM 42, thereby controlling the signal processing unit 20a and the entire imaging apparatus in an integrated manner, and executes various operations for the control. In addition, when the determination signal indicates that the data amount of the variable length compressed data exceeds the prescribed data amount, the CPU 29a controls each block to use the fixed length compressed data.

  The video output encoder 30 converts the output image data supplied from the resolution converter 27 and the JPEG engine 28 into a format corresponding to a device connected to the video output terminal 30a, and converts the converted image data to the video output terminal. Output from 30a. Further, the video output encoder 30 converts the output image data into display data for monitor display and outputs the display data to the display unit 44 described later.

  The SDRAM controller 31 is an interface to the SDRAM 41, and includes an address decoder and the like, and controls write and read operations of the SDRAM 41 according to a control signal from the CPU 29a.

  As described above, the SRAM 33 is a memory that temporarily stores the fixed-length compressed data generated by the fixed-length RAW compression unit 24 during imaging.

  The SDRAM 41 is a volatile memory used as a work area for data processing or the like in the signal processing unit 20. The SDRAM 41 is provided with a capture data area 411, a JPEG code area 412, a CPU work area 413, and the like. The capture data area 411 is an area for temporarily storing variable-length compressed data generated by the variable-length RAW compressing unit 22 and fixed-length compressed data generated by the fixed-length RAW compressing unit 24. The JPEG code area 412 is an area for temporarily storing image data encoded by the JPEG engine 28, data used in the encoding / decoding process, and the like. The CPU work area 413 is an area for temporarily storing data used in the processing of the CPU 29a.

  The ROM 42 stores programs executed by the CPU 29a and various data. As the ROM 42, for example, a nonvolatile memory such as an EPROM flash memory may be used.

  In the recording / reproducing unit 43, for example, a recording medium such as a flash memory, an optical disk, or a magnetic tape is used. The recording / reproducing unit 43 records the image data output from the signal processing unit 20 on a recording medium. Further, the recording / reproducing unit 43 performs processing of reading out image data recorded on the recording medium and supplying the read image data to the signal processing unit 20. Note that the recording medium used in the recording / reproducing unit 43 may be detachable or may not be detachable.

  The display unit 44 displays a camera-through image captured by the imaging apparatus 10 and a captured image recorded in the recording / playback unit 43. The display unit 44 also displays a menu for setting the imaging device 10 and the like.

  The operation unit 45 includes operation buttons and a touch panel provided on the screen of the display unit 44. The operation unit 45 generates an operation signal corresponding to a user operation such as a shutter operation and outputs the operation signal to the CPU 29a of the signal processing unit 20.

<4. Operation of Second Embodiment>
When the shutter operation is performed by the operation unit 45, the imaging device 10a performs the processing of the flowchart illustrated in FIG. In FIG. 9, in step ST11, the CPU 29a of the signal processing unit 20a performs an imaging operation. The CPU 29a controls the imaging unit 12 to perform imaging with a desired exposure time, stores the captured image in the capture data area 411 of the SDRAM 41, and proceeds to step ST2.

  FIG. 10 shows a signal path when the imaging operation is performed. The imaging signal generated by the imaging unit 12 by performing imaging with a desired exposure time is subjected to noise removal processing or the like by the AFE unit 13, and then converted into a digital signal to be preprocessed by the signal processing unit 20a. Supplied to the unit 21. The camera signal preprocessing unit 21 performs defect correction processing, shading correction processing, and the like, and outputs the processed signal to the variable length RAW compression unit 22a and the fixed length RAW compression unit 24 as RAW data. The variable length RAW compression unit 22a compresses the RAW data supplied from the camera signal preprocessing unit 21 using a variable length encoding method. Further, the variable length RAW compression unit 22 a stores the variable length compressed data generated by performing the compression process in the capture data area 411 of the SDRAM 41 via the bus 32 or the SDRAM controller 31. Further, the variable length RAW compression unit 22a does not output the variable length compressed data when the data amount of the variable length compressed data exceeds the specified data amount, and the data amount of the variable length compressed data is specified to the CPU 29a. It is notified by a determination signal that the amount of data is exceeded.

  The fixed-length RAW compression unit 24 compresses the RAW data supplied from the camera signal preprocessing unit 21 using a fixed-length encoding method. Further, the fixed-length RAW compression unit 24 stores the fixed-length compressed data generated by performing the compression process in the SRAM 33.

  In step ST12, the CPU 29a determines whether or not the data amount of the variable length compressed data exceeds a prescribed data amount. When the determination signal from the variable length RAW compression unit 22a does not indicate that the CPU 29a exceeds the prescribed data amount, the CPU 29a proceeds to step ST13. Further, the CPU 29a proceeds to step ST17 when the determination signal from the variable length RAW compression unit 22a indicates that it exceeds the prescribed data amount. For example, when the imaging mode of the imaging apparatus 10a is set to the continuous shooting imaging mode and the number of continuous shooting is “third image”, the determination signal from the variable length RAW compression unit 22a exceeds the specified data amount. Suppose that In this case, the process proceeds to step ST17 with the variable length compressed data for two sheets stored in the capture data area 411 of the SDRAM 41 and the third fixed length compressed data stored in the SRAM 33.

  In step ST13, the CPU 29a determines whether or not the imaging is finished. When the imaging mode is set to the continuous shooting mode, the CPU 29a determines whether or not variable-length compressed data corresponding to a preset number of continuous shots is stored in the capture data area 411 of the SDRAM 41. The CPU 29a returns to step ST11 when the variable-length compressed data for the number of continuous shots is not stored in the capture data area 411 of the SDRAM 41. Further, when the variable-length compressed data for the number of continuous shots is stored in the capture data area 411 of the SDRAM 41, the CPU 29a determines that the imaging operation is finished and proceeds to step ST14.

  When the imaging mode is set to the single-shot imaging mode, the variable length compressed data of the captured image when the shutter operation is performed is stored in the capture data area 411 of the SDRAM 41 by performing the process of step ST11. . Accordingly, since the variable length compressed data for one sheet is stored in the capture data area 411 of the SDRAM 41, the CPU 29a determines that the imaging operation is finished and proceeds to step ST14.

  In step ST14, the CPU 29a performs compression rate conversion. The CPU 29a controls the variable-length RAW decompression unit 23, the fixed-length RAW compression unit 24, and the SDRAM controller 31 to convert one piece of variable-length compressed data stored in the capture data area 411 of the SDRAM 41 into fixed-length compressed data. The conversion proceeds to step ST15.

  The signal path when the compression rate conversion operation is performed is the same as in FIG. That is, the CPU 29a supplies one variable-length compressed data stored in the capture data area 411 of the SDRAM 41 to the variable-length RAW expansion unit 23 via the SDRAM controller 31 or the bus 32.

  The variable length RAW decompression unit 23 decompresses variable length compressed data read from the capture data area 411 of the SDRAM 41. Further, the variable length RAW decompression unit 23 outputs the RAW data obtained by performing the decompression process to the fixed length RAW compression unit 24.

  The fixed-length RAW compression unit 24 compresses the RAW data supplied from the variable-length RAW expansion unit 23 using a fixed-length encoding method. The fixed-length RAW compression unit 24 stores the fixed-length compressed data obtained by performing the compression process in the capture data area 411 of the SDRAM 41 via the bus 32 or the SDRAM controller 31.

  As described above, when the compression rate conversion operation is performed, one fixed-length compressed data can be stored in the capture data area 411 of the SDRAM 41.

  In step ST15, the CPU 29a performs a RAW development process similar to that in step ST4. The CPU 29 a controls the fixed-length RAW decompression unit 25 and the SDRAM controller 31 to read out the fixed-length compressed data corresponding to the portion where the RAW development processing is performed from the capture data area 411 of the SDRAM 41 and supplies the fixed-length compressed data to the fixed-length RAW decompression unit 25. . Further, the CPU 29 a controls the camera signal processing unit 26 to perform RAW development processing of the RAW data generated by performing the expansion processing in the fixed-length RAW expansion unit 25. Further, the CPU 29a causes the resolution conversion unit 27 to perform resolution conversion, the JPEG engine 28 to perform compression encoding processing, or the like on the image data after RAW development processing in accordance with settings or the like registered in advance by the user. Proceed to ST16.

  The CPU 29a records the image data after the RAW development processing or the image data on which the resolution conversion or the compression encoding processing has been performed on the image data after the RAW development processing on the recording medium of the recording / playback unit 43.

  In step ST16, the CPU 29a determines whether or not processing for the number of captured images has been completed. When the imaging mode is set to the continuous shooting mode, the CPU 29a performs the compression rate conversion operation and the RAW development processing for each of the variable length compressed data for the number of continuous shots stored in the capture data area 411 of the SDRAM 41. It is determined whether or not it has been performed. The CPU 29a returns to step ST14 when variable-length compressed data for which the compression rate conversion operation and the RAW development processing are not completed remains, reads new variable-length compressed data, and performs the processes of steps ST14 and ST15. Further, the CPU 29a ends the processing when the compression rate conversion and the RAW development processing are performed for each of the variable length compressed data for the number of continuous shots.

  When the shooting mode is set to the single shooting mode, only one variable-length compressed data is stored in the capture data area 411 of the SDRAM 41. Therefore, when the processes of step ST3 and step ST4 are performed, the process ends because there is no variable length compressed data for which the compression rate conversion operation and the RAW development process have not been completed.

  Next, since the determination signal from the variable-length RAW compression unit 22a indicates that the amount of data exceeds the prescribed amount, when the process proceeds from step ST12 to step ST17, the CPU 29a compresses the stored variable-length compressed data. Perform rate conversion. The CPU 29a stops the imaging operation when the determination signal indicates that the prescribed data amount is exceeded, and ends the process of storing the variable-length compressed data in the variable-length compressed data storage unit. Thereafter, the compression rate conversion of the variable length compressed data temporarily stored in the SDRAM 41 is performed. The CPU 29a controls the variable-length RAW decompression unit 23, the fixed-length RAW compression unit 24, and the SDRAM controller 31 to convert one piece of variable-length compressed data stored in the capture data area 411 of the SDRAM 41 into fixed-length compressed data. The conversion proceeds to step ST18.

  In step ST18, the CPU 29a performs RAW development processing. The CPU 29a controls the fixed length RAW expansion unit 25 and the SDRAM controller 31 to read and expand the fixed length compressed data stored in the capture data area 411 of the SDRAM 41. Further, the CPU 29a controls the camera signal processing unit 26 to perform RAW development processing of the RAW data generated by the fixed length RAW expansion unit 25. Further, the CPU 29a causes the resolution conversion unit 27 to perform resolution conversion, the JPEG engine 28 to perform compression encoding processing, or the like on the image data after RAW development processing in accordance with settings or the like registered in advance by the user. Proceed to ST19.

  The CPU 29a records the image data after the RAW development processing or the image data on which the resolution conversion or the compression encoding processing has been performed on the image data after the RAW development processing on the recording medium of the recording / playback unit 43.

  In step ST19, the CPU 29a determines whether or not the processing of the stored variable length encoded data is completed. The CPU 29a returns to step ST17 when there remains a captured image for which the compression rate conversion and the RAW development processing have not been completed, reads the variable length RAW data of the new captured image, and performs the processing of step ST17 and step ST18. Further, the CPU 29a proceeds to step ST20 when it performs compression rate conversion and RAW development processing for each of the held variable-length compressed data.

  In step ST <b> 20, the CPU 29 a performs RAW development processing using the fixed-length compressed data stored in the SRAM 33.

  FIG. 11 shows a signal path when RAW development processing is performed in step ST20. The CPU 29a calculates an address at which fixed-length compressed data corresponding to a portion where RAW development processing is performed is stored. Based on this address, the data of the portion to be developed is read from the fixed-length compressed data stored in the SRAM 33 and supplied to the fixed-length RAW expansion unit 25.

  The fixed-length RAW decompression unit 25 performs decompression processing on the fixed-length compressed data read from the SRAM 33. The fixed-length RAW expansion unit 25 outputs RAW data obtained by performing the expansion process to the camera signal processing unit 26.

  The camera signal processing unit 26 performs RAW development processing using the RAW data supplied from the fixed-length RAW expansion unit 25. That is, the RAW development processing for the entire screen is partially performed sequentially.

  Note that the image data after the RAW development processing may be recorded as it is on the recording medium by the recording / reproducing unit 43 as described above, or recorded after the image data is processed by the resolution converting unit 27 and / or the JPEG engine 28. You may record on a medium. Alternatively, the image data after the RAW development processing or the image data may be processed by the resolution conversion unit 27 and then encoded by the JPEG engine 28 and stored in the JPEG code area 412 of the SDRAM 41.

  As described above, when the data amount of the variable length compressed data exceeds the specified value, the temporary holding of the variable length compressed data is stopped, and the RAW development processing is performed using the fixed length encoded data. In this way, when the data amount of the variable length compressed data becomes larger than the predetermined data amount, the RAW data indicating the captured image is fixed even if the process of temporarily storing the variable length compressed data is stopped. Long-compressed and stored in the SRAM 33. Therefore, the imaged data is not lost. Furthermore, RAW development processing can be performed by using fixed-length compressed data stored in the SRAM 33. Further, if the capacity of the SRAM 33 is set as the amount of fixed-length compressed data for a plurality of sheets, even if the amount of data exceeds a specified amount during continuous shooting, the fixed-length compressed data of the remaining captured images is stored. You can also keep it.

  It should be noted that the present invention should not be construed as being limited to the above-described embodiments of the invention. The embodiment of the present invention discloses the present invention in the form of exemplification, and it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the gist of the present invention. That is, in order to determine the gist of the present invention, the claims should be taken into consideration.

  In the imaging apparatus and the imaging method of the present invention, variable length compression is performed on the RAW data, and the variable length compressed data generated by the compression processing is temporarily stored in the variable length compressed data storage unit. The In addition, the fixed-length compression data generated by the compression processing is fixed by performing the compression processing of the fixed-length compression method on the RAW data generated by performing the decompression processing of the stored variable-length compression data. Temporarily stored in the long compressed data storage unit. Further, the stored fixed-length compressed data is expanded for each area obtained by dividing the entire screen in the line direction, and image data generated by performing this expansion processing is developed.

  For this reason, RAW data generated by performing single-shot imaging or continuous shooting imaging is temporarily stored as variable-length compressed data, so that a high-speed imaging operation can be performed. Also, since RAW data generated by decompressing variable-length compressed data is temporarily stored as fixed-length compressed data, the fixed-length compressed data is decompressed and developed for each area obtained by dividing the entire screen in the line direction. By performing the above, it is possible to perform RAW development processing at high speed even if the scale of the line memory, delay line, etc. is reduced. Therefore, it is suitable for a digital camera or the like that can perform continuous shooting imaging as well as single vehicle imaging.

It is a figure which shows the structure of the conventional imaging device. It is a block diagram which shows the structure of 1st Embodiment. 5 is a flowchart illustrating processing when a shutter operation is performed in the first embodiment. It is a figure which shows a signal path | route when imaging operation is performed. It is a figure which shows a signal path | route when compression rate conversion operation | movement is performed. It is a figure for demonstrating RAW image development processing. It is a figure which shows the signal path | route when a development process is performed. It is a block diagram which shows the structure of 2nd Embodiment. 9 is a flowchart illustrating processing when a shutter operation is performed in the second embodiment. It is a figure which shows a signal path | route when imaging operation is performed. It is a figure which shows a signal path | route when RAW image development processing is performed.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10,10a ... Imaging device, 11 ... Imaging optical system block, 12 ... Imaging part, 13 ... Analog front end (AFE) part, 20, 20a ... Signal processing part, 21 ... Camera signal preprocessing unit, 22, 22a ... compression unit, 23 ... variable length RAW expansion unit, 24 ... fixed length RAW compression unit, 25 ... fixed length RAW expansion unit, 26 ... Camera signal processing unit, 27... Resolution conversion unit, 28... JPEG engine, 29, 29a... CPU, 30... Video output encoder, 30a. 32 ... Bus, 33 ... SRAM, 41 ... SDRAM, 42 ... ROM, 43 ... Recording / reproducing unit, 44 ... Display unit, 45 ... Operating unit, 411 ... Capture data area, 412, · JPEG code area, 413 ... work area

Claims (6)

A variable-length compression unit that performs compression processing by a variable-length compression method on image data obtained by digitally converting the output from the image sensor, and generates variable-length compressed data;
A variable-length compressed data storage unit for temporarily storing the variable-length compressed data;
A variable length decompression unit for decompressing the stored variable length compressed data;
A fixed-length compression unit that performs compression processing by a fixed-length compression method on the image data generated by performing the expansion process in the variable-length RAW expansion unit, and generates fixed-length compressed data;
A fixed-length compressed data storage unit for temporarily storing the fixed-length compressed data;
A fixed-length decompression unit that divides the entire screen in the line direction, and performs decompression processing of the fixed-length compressed data corresponding to the partitioned area;
An image pickup apparatus comprising: a development processing unit that performs development processing for each of the divided areas using image data generated by performing expansion processing by the fixed-length expansion unit.   The variable length compression unit, the variable length compressed data storage unit, the variable length expansion unit, and the fixed length compression unit, the fixed length compression data storage unit, and the fixed length expansion unit are connected via a bus. The imaging device according to claim 1. A second fixed-length compressed data storage unit connected to the fixed-length compressing unit and the fixed-length expanding unit without going through the bus;
The variable length compression unit determines whether or not the variable length compressed data exceeds a predetermined amount of data,
The fixed-length compression unit compresses image data obtained by digitally converting the output from the image sensor by a fixed-length compression method, generates fixed-length compressed data, and performs the second fixed-length compression. Store in the data storage unit,
When the variable length compression unit determines that the variable length compressed data exceeds a predetermined amount of data, the image data that has undergone compression processing is subjected to the second fixed length by the fixed length decompression unit. The imaging apparatus according to claim 2, wherein decompression processing of fixed-length compressed data stored in a compressed data storage unit is performed for each of the divided areas, and image data generated by performing the decompression processing is used. The variable length compression unit performs compression processing of image data obtained by performing single shooting imaging or continuous shooting imaging, and stores the generated variable length compressed data in the variable length compressed data storage unit,
The imaging apparatus according to claim 1, wherein the variable length decompression unit performs decompression processing of the stored variable length compressed data after the single-shot imaging or continuous-shot imaging.   The variable length compressed data storage unit and the fixed length compressed data storage unit are configured by the same storage unit, the variable length compressed data is stored in a first storage area of the storage unit, and the second of the storage unit The imaging apparatus according to claim 1, wherein the fixed-length compressed data is stored in a storage area. A step of performing a variable length compression method compression processing in a variable length compression unit on image data obtained by digitally converting the output from the image sensor, and generating variable length compressed data;
Temporarily storing the variable-length compressed data in a variable-length compressed data storage unit;
Performing a decompression process of the stored variable-length compressed data in a variable-length decompression unit;
A step of performing a fixed-length compression method compression processing on a fixed-length compression unit for image data generated by performing the expansion process on the variable-length expansion unit, and generating fixed-length compressed data;
A step of temporarily storing the fixed-length compressed data in a fixed-length compressed data storage unit;
The entire screen is divided in the line direction, and a fixed length decompression unit performs a decompression process of the fixed length compressed data corresponding to each of the divided areas;
And a step of performing development processing for each of the divided areas using image data generated by performing extension processing in the fixed-length extension unit.

Images