JP2008193464A - Image processing device and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means capable of more quickly executing compression processing of image data. <P>SOLUTION: This image processing device includes a first compression processing part, a second compression processing part and a control part. The first compression processing part compresses data of an input image and generates first compressed data. A second compression processing part compresses the data of the input image and generates second compressed data. The control part divides an image of one frame into two areas and respectively makes the first compression processing part and the second compression processing part perform parallel processing of the divided image data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an imaging apparatus that perform compression processing of image data.

In an image processing apparatus such as an electronic camera, it is common to perform compression processing on image data in order to increase recording efficiency. Here, in the compression processing in the image processing apparatus, from the viewpoint of easily grasping the number of remaining frames, fixed-length compression that makes the amount of compressed data for one frame substantially constant by adjusting the scale factor is often used. Note that Patent Document 1 discloses an example of the configuration of an electronic camera that performs fixed-length compression processing.
JP-A-9-219807

By the way, from the viewpoint of improving the continuous shooting speed of the imaging apparatus and speeding up the image processing, there is still a demand for further speeding up the compression processing for image data.
The present invention solves the above-mentioned problems of the prior art. An object of the present invention is to provide a means capable of executing image data compression processing more quickly.

  An image processing apparatus according to a first aspect includes a first compression processing unit, a second compression processing unit, and a control unit. The first compression processing unit compresses the input image data to generate first compressed data. The second compression processing unit compresses the input image data to generate second compressed data. The control unit divides the image of one frame into two regions, and causes the first compression processing unit and the second compression processing unit to process the data of the divided images in parallel.

In a second aspect based on the first aspect, the image processing apparatus further includes a memory for temporarily recording data and a data transfer unit. The data transfer unit records the first compressed data and the second compressed data in a common recording area allocated for one frame on the memory.
In a third aspect based on the second aspect, the data transfer unit records the first compressed data in the forward direction from the start address of the recording area, and the second compressed data in the reverse direction from the end address of the recording area. Record.

In a fourth aspect based on the third aspect, the data transfer unit reverses the arrangement order of the second compressed data and records it in the recording area.
In a fifth aspect based on the first aspect, the control unit calculates a parameter at the time of recompression based on the compression results of the first compression processing unit and the second compression processing unit.
Here, the imaging apparatus may be configured by adding an imaging unit that captures a subject image and generates image data to the above-described invention. In addition, what is expressed by converting the contents of each of the above inventions into an image processing method or a computer program is also effective as a specific aspect of the present invention.

  In the image processing apparatus of the present invention, image data for one frame can be processed in parallel by the first compression processing unit and the second compression processing unit, and the compression processing can be executed more quickly.

  FIG. 1 is a block diagram showing the configuration of the electronic camera of this embodiment. The electronic camera includes an imaging optical system 11, an imaging element 12, an analog signal processing unit 13, an A / D conversion unit 14, a buffer memory 15, an image processing unit 16, a first compression processing unit 17 and a second compression unit. A compression processing unit 18, a DMA (Direct Memory Access) control unit 19, a recording I / F 20, an operation member 21 and a release button 22, a ROM 23, a CPU 24 and a system bus 25 are provided.

  Here, the buffer memory 15, the image processing unit 16, the first compression processing unit 17, the second compression processing unit 18, the DMA control unit 19, the recording I / F 20, and the CPU 24 are respectively connected via a system bus 25. . The system bus 25 has a plurality of data buses (in FIG. 1, illustration of individual data buses is omitted). Note that the image sensor 12, the analog signal processing unit 13, the A / D conversion unit 14, the operation member 21, the release button 22, and the ROM 23 are connected to the CPU 24, respectively.

The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens. For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.
The imaging element 12 is disposed on the image space side of the imaging optical system 11. The image sensor 12 captures a recording image (main image) at the time of release, and photoelectrically converts the light beam that has passed through the imaging optical system 11 to generate an analog image signal. The output of the image sensor 12 is connected to an analog signal processing unit 13.

The analog signal processing unit 13 is an analog front-end circuit that performs analog signal processing on the output of the image sensor 12. The analog signal processing unit 13 performs correlated double sampling, image signal gain adjustment, and the like. The output of the analog signal processing unit 13 is connected to the A / D conversion unit 14.
The A / D converter 14 performs A / D conversion on the image signal output from the analog signal processor 13. The output of the A / D converter 14 is connected to the buffer memory 15.

The buffer memory 15 buffers the main image data output from the A / D converter 14. The buffer memory 15 temporarily records the main image data before and after the compression process.
The image processing unit 16 performs various types of image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on image data for one frame.

The first compression processing unit 17 is a processor that compresses the data of the main image after image processing in the JPEG (Joint Photographic Experts Group) format. The first compression processing unit 17 includes a code amount measuring unit 17a that measures the amount of compressed data.
Here, the 1st compression process part 17 performs the following series of compression processes. First, the first compression processing unit 17 performs DCT (discrete cosine transform) processing on the image to be processed to obtain DCT coefficients. This DCT conversion is executed in units of 8 × 8 pixel blocks. Secondly, the first compression processing unit 17 performs DCT coefficient quantization processing based on a quantization table described later. Thirdly, the first compression processing unit 17 performs Huffman encoding processing on the quantized value, and generates compressed data of the main image.

The second compression processing unit 18 is a processor having the same configuration as that of the first compression processing unit 17 and includes a code amount measurement unit 18a inside. The second compression processing unit 18 is not redundantly described with the first compression processing unit 17. Note that the first compression processing unit 17 and the second compression processing unit 18 can perform fixed-length compression on one frame of main image data by parallel processing.
The DMA control unit 19 controls DMA transfer of image data between the modules for each data bus based on an instruction from the CPU 24. That is, the DMA control unit 19 can simultaneously transfer data to different modules for each data bus. The DMA control unit 19 can also reverse the order of data arrangement when transferring data between modules.

  A connector for connecting the recording medium 26 is formed in the recording I / F 20. The recording I / F 20 executes data writing / reading with respect to the recording medium 26 connected to the connector. The recording medium 26 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 26.

  The operation member 21 includes, for example, a command dial, a cross-shaped cursor key, a determination button, and the like. The operation member 21 receives various setting inputs of the electronic camera from the user. For example, the user performs (a) selection of a compression mode (for example, “FINE”, “NORMAL”, “BASIC”, etc.) relating to the compression ratio of the main image, or (b) switching of the recording mode with the operation member 21. be able to. In the recording mode switching described above, the user can select a first recording mode for performing normal image processing and a second recording mode for recording RAW data (raw image data).

The release button 22 receives an instruction input of a release timing (start of exposure of the main image) from the user.
The ROM 23 stores a sequence program for controlling the electronic camera, a standard quantization table, a scale factor table, and the like. Here, the standard quantization table stores numerical values that define quantization steps for each DCT coefficient. The numerical values of this standard quantization table are set so as to greatly reduce the amount of high-frequency component data that is inconspicuous even if lost due to human visual characteristics.

  On the other hand, in the scale factor table, the correspondence between the value of the scale factor (coefficient related to the compression rate) at the time of compression processing and the amount of compressed data is recorded for each of the plurality of sample image data. Here, the value of the scale factor table is obtained by repeating the compression process while gradually changing the value of the scale factor for each sample image data after the DCT process. For the same image data, the data compression ratio tends to increase as the scale factor increases.

The CPU 24 is a processor that performs overall control of the electronic camera. CPU24 controls operation | movement of each part of an electronic camera according to said sequence program.
In addition, the CPU 24 in the present embodiment divides one frame of the main image into two regions, and the divided main image data is respectively processed in parallel in the first compression processing unit 17 and the second compression processing unit 18. Compress. Further, the CPU 24 performs determination of a scale factor, calculation of a quantization table value, and the like regarding the compression processing of the data of the main image.

Hereinafter, the imaging operation of the electronic camera in the present embodiment will be described. In the following example, description will be made on the assumption that the electronic camera is set to the first recording mode by the user's operation.
When the release button 22 is fully pressed by the user, the CPU 24 drives the image pickup device 12 and executes the image pickup of the main image. The image signal of the main image read from the image sensor 12 passes through the analog signal processing unit 13 and the A / D conversion unit 14 in a pipeline manner and is buffered in the buffer memory 15. The data of the main image is recorded again in the buffer memory 15 after various image processing is performed by the image processing unit 16.

Then, the CPU 24 causes the first compression processing unit 17 and the second compression processing unit 18 to perform compression processing, and compresses the data of the main image with a fixed length. Hereinafter, the compression processing in the electronic camera according to the present embodiment will be described in detail with reference to the flowchart of FIG.
Step 101: The CPU 24 divides the data of the main image to be compressed into two, and assigns the compression processing of the divided data to the first compression processing unit 17 and the second compression processing unit 18, respectively. In the present embodiment, the CPU 24 divides the main image vertically into two parts from the center. Then, the CPU 24 assigns the compression process for the upper half of the main image to the first compression processing unit 17 and the compression process for the lower half of the main image to the second compression processing unit 18.

Step 102: The CPU 24 determines a compression target data amount based on the current compression mode setting. Then, the CPU 24 secures a recording area for one frame corresponding to the compression target data amount on the buffer memory 15. Note that first compressed data and second compressed data, which will be described later, are written in the recording area secured in S102.
Step 103: The CPU 24 refers to the scale factor table in the ROM 23 and sets the scale factor based on the compression target data amount (S102).

  Step 104: The CPU 24 multiplies the value of the standard quantization table by the scale factor to obtain the value of the quantization table used for the compression process. Then, the CPU 24 outputs the data of the quantization table value to the first compression processing unit 17 and the second compression processing unit 18, respectively. Therefore, the first compression processing unit 17 and the second compression processing unit 18 perform compression processing of the main image based on the same quantization table value.

Step 105: The CPU 24 operates the first compression processing unit 17, the second compression processing unit 18, and the DMA control unit 19 to execute the compression processing of the main image data. The compression processing in S105 is performed by parallel processing of the first compression processing unit 17 and the second compression processing unit 18.
First, the compression processing of the main image data by the first compression processing unit 17 will be described.
First, the DMA control unit 19 DMA-transfers data corresponding to the upper half of the image among the main image data recorded in the buffer memory 15 to the first compression processing unit 17. The DMA control unit 19 sequentially reads out image data for each block of 8 × 8 pixels starting from the upper left side of the upper half image. Note that the DMA control unit 19 reads in order from the leftmost block to the rightmost block of the image. When the reading of the rightmost block is completed, the DMA control unit 19 performs the same reading as described above for the blocks arranged horizontally one step below. FIG. 3 schematically shows a method for reading image data.

Second, the first compression processing unit 17 compresses the upper half of the image data in units of blocks to generate first compressed data. Further, the code amount measuring unit 17a of the first compression processing unit 17 sequentially outputs the data amount of the first compressed data to the CPU 24.
Third, the DMA control unit 19 DMA-transfers the first compressed data having a predetermined length to the recording area (S102) of the buffer memory 15 in the output order.

  Here, the DMA control unit 19 starts recording the first compressed data from the start address of the recording area. Then, the DMA control unit 19 sequentially adds the recording address value of the head data during the DMA transfer of the first compressed data. Thus, the first compressed data is sequentially recorded by a predetermined length in the forward direction from the start address of the recording area. FIG. 4 schematically shows a compressed data recording method.

Next, the compression processing of the main image data by the second compression processing unit 18 will be described. The compression processing of the second compression processing unit 18 is performed almost simultaneously with the compression processing of the first compression processing unit 17 described above.
First, the DMA control unit 19 DMA-transfers data corresponding to the lower half of the image among the main image data recorded in the buffer memory 15 to the second compression processing unit 18. The DMA control unit 19 sequentially reads out the image data for each 8 × 8 pixel block starting from the upper left side of the lower half image (see FIG. 3). In addition, since the reading method of each block is the same as that of the case of the above-mentioned 1st compression process part 17, duplicate description is abbreviate | omitted.

Secondly, the second compression processing unit 18 compresses the lower half image data in units of blocks to generate second compressed data. The code amount measuring unit 18a of the second compression processing unit 18 sequentially outputs the data amount of the second compressed data to the CPU 24.
Thirdly, the DMA control unit 19 DMA-transfers the second compressed data having a predetermined length to the recording area (S102) of the buffer memory 15.

  Here, the DMA control unit 19 starts recording the second compressed data from the end address side of the recording area. At this time, the DMA control unit 19 reverses the arrangement order of the second compressed data during the DMA transfer, and records so that the most significant bit before the inversion is located at the last address. Then, the DMA control unit 19 sequentially subtracts the recording address value of the head data during the DMA transfer of the second compressed data. As a result, the second compressed data is sequentially recorded by a predetermined length in the reverse direction from the last address (see FIG. 4). When the total data amount of the first compressed data and the second compressed data is smaller than the capacity (buffer size) of the recording area, an empty area is generated in the middle part of the recording area.

  As an example, consider a case where a data string “0101101101” is output from the second compression processing unit 18 as the second compressed data. The DMA control unit 19 inverts the arrangement order of the second compressed data and transfers it to the buffer memory 15 as “10111010”. The data string “10111010” is recorded such that the least significant bit is located at the end address of the recording area. Therefore, when the data is read in the reverse direction from the last address of the recording area, the original data string of the second compressed data (“010111101”) is correctly output.

Step 106: Based on the output of the code amount measuring units 17a and 18a, the CPU 24 determines that the total compressed data amount (total of the first compressed data and the second compressed data) is the capacity of the recording area (S102) during the compression processing in S105. It is determined whether it is over (buffer over) or not.
If the total compressed data amount exceeds the capacity of the recording area (YES side), the CPU 24 proceeds to S107 to perform the compression process again. On the other hand, when the total amount of compressed data falls within the capacity of the recording area (NO side), the CPU 24 proceeds to S108.

Step 107: The CPU 24 corrects the scale factor so as to increase the compression rate by any of the following means (1) to (3). Thereafter, the CPU 24 returns to S104 and repeats the above operation.
(1) The CPU 24 causes the DMA control unit 19 to stop transferring compressed data when the total amount of compressed data exceeds the capacity of the recording area. On the other hand, the CPU 24 causes the first compression processing unit 17 and the second compression processing unit 18 to compress the data of the main image to the end, and obtains the final total compressed data amount from the output of the code amount measuring units 17a and 18a. Ask. In this case, the compressed data after the buffer over is used only for code amount measurement, and the compressed data is not recorded. Then, the CPU 24 obtains a scale factor based on the final total compressed data amount so that the data of the main image falls within the compression target data amount.

  For example, the CPU 24 estimates sample image data that approximates the main image to be compressed from the combination of the current scale factor and the final total compressed data amount. Then, the CPU 24 calculates the scale factor again using the scale factor table corresponding to the estimated sample image data. In this case, based on the current compression result, the CPU 24 can estimate the scale factor used for the next compression with high accuracy.

(2) The CPU 24 stops all the compression processing by the first compression processing unit 17 and the second compression processing unit 18 when the total amount of compressed data exceeds the capacity of the recording area. Then, the CPU 24 obtains a scale factor according to the following procedure based on the output of the code amount measuring units 17a and 18a and the amount of uncompressed remaining data in the main image data.
First, the CPU 24 obtains the data amount of the remaining data.

Secondly, the CPU 24 obtains the respective data amounts of the first compressed data and the second compressed data recorded in the recording area based on the outputs of the code amount measuring units 17a and 18a.
Thirdly, the CPU 24 assumes that the remaining data is compressed as in the case of the first compressed data and the second compressed data, and estimates the compressed data amount for the remaining data.
Specifically, the CPU 24 determines the actual data in the first compression processing unit 17 from the data amount of the compressed upper half image and the data amount of the first compressed data for the remaining data corresponding to the upper half of the main image. Determine the compression ratio. Then, the CPU 24 obtains an estimated value of the compressed data amount by multiplying the data amount of the remaining data by the above compression rate. Note that the CPU 24 calculates an estimated value of the compressed data amount for the remaining data corresponding to the lower half of the main image in the same process as described above.

  Fourthly, the CPU 24 obtains a final estimated value of the total compressed data amount from the sum of the total estimated value of the compressed data amount and the capacity of the recording area. Then, the CPU 24 resets the scale factor based on the final estimated value of the total compressed data amount so that the data of the main image is within the compression target data amount. Note that the calculation of the scale factor in this case is the same as in the case of the above (1), and therefore a duplicate description is omitted.

Also in the case of (2), based on the current compression result, the CPU 24 can estimate the scale factor used for the next compression with high accuracy. In the case of (2), since it is not necessary to perform the compression process to the end, the CPU 24 can shorten the processing time as compared with the case of (1).
(3) The CPU 24 stops all the compression processing by the first compression processing unit 17 and the second compression processing unit 18 when the total amount of compressed data exceeds the capacity of the recording area. And CPU24 calculates | requires a scale factor so that uncompressed remaining data can also be compressed. As an example, considering the case where the ratio of uncompressed remaining data to the entire data of the main image is 4: 1, the CPU 24 reduces the compression ratio to 125% ((4 + 1) / 4) as compared with the compression condition in S105. Reset the scale factor to increase. In the case of (3), it is not necessary to perform the compression process to the end, so the CPU 24 can shorten the processing time compared to the case of (1).

Step 108: The CPU 24 determines whether or not the number of executions of the compression process (S105) exceeds a threshold value. If the number of executions of the compression process exceeds the threshold (YES side), the CPU 24 proceeds to S111 and terminates the compression process for the main image data. On the other hand, when the number of executions of the compression process is equal to or less than the threshold (NO side), the CPU 24 proceeds to S109.
Step 109: The CPU 24 determines whether or not the total compressed data amount (S106) falls within an allowable range having the compression target data amount as an upper limit. If the total compressed data amount is within the allowable range (YES side), the CPU 24 proceeds to S111. On the other hand, when the total compressed data amount is below the allowable range (NO side), the CPU 24 proceeds to S110 to perform the compression process again.

  Step 110: The CPU 24 corrects the scale factor so that the compression rate becomes low. In this case, the CPU 24 obtains the capacity of the free area in the recording area based on the data amounts of the first compressed data and the second compressed data. Then, the CPU 24 resets the scale factor so that the total amount of compressed data increases according to the capacity of the free area. Thereafter, the CPU 24 returns to S104 and repeats the above operation.

Step 111: The CPU 24 records the first compressed data and the second compressed data recorded in the recording area of the buffer memory 15 on the recording medium 26. At this time, the DMA control unit 19 transfers each compressed data in accordance with an instruction from the CPU 24 as follows.
First, the DMA control unit 19 reads the first compressed data from the start address of the recording area in the forward direction and transfers it to the recording medium 26. Here, the first compressed data is recorded on the recording medium 26 in the output order from the first compression processing unit 17.

  Next, when reading of the first compressed data is completed, the DMA control unit 19 reads the second compressed data in the reverse direction from the last address of the recording area and transfers it to the recording medium 26. The arrangement order of the second compressed data after the transfer is in the order of the output from the second compression processing unit 18 as described above. The first compressed data and the second compressed data are continuously recorded on the recording medium 26 as one data string. As a result, the compressed data of the main image for one frame is recorded on the recording medium 26 in the same state as when compressed by one compression processor. Above, description of the imaging operation of the electronic camera in this embodiment is complete | finished.

Hereinafter, the effect of this embodiment is demonstrated. In the electronic camera of this embodiment, the first compression processing unit 17 and the second compression processing unit 18 compress one frame of the main image by parallel processing. Therefore, compared with the case where image data is compressed by a single compression processor, the speed of the compression process can be increased in this embodiment.
In the electronic camera of this embodiment, the first compressed data of the first compression processing unit 17 and the second compressed data of the second compression processing unit 18 are temporarily recorded in a common recording area of the buffer memory 15. . Therefore, in this embodiment, the recording efficiency of compressed data in the buffer memory 15 can be increased. As an example, consider a case where the compression code amount of one compression processing unit is small and the compression code amount of the other compression processing unit is large. Here, if the recording areas of the buffer memory 15 are separately allocated to the first compression processing section 17 and the second compression processing section 18, the buffer size of each recording area is set with a large code amount in consideration of the worst case. Need to match. That is, if a recording area is prepared for each compression processing unit, a buffer size larger than the compression target data amount is required.

On the other hand, in the configuration in which each compression processing unit shares a recording area as in this embodiment, if the compression code amount of one compression processing unit is small, the compression code amount of the other compression processing unit is increased by that amount. Can record a lot. Therefore, in the configuration of the present embodiment, it is sufficient to secure a recording area having a size corresponding to the compression target data amount in the buffer memory 15.
In particular, in the electronic camera of this embodiment, the first compressed data is recorded in the forward direction from the start address of the recording area, while the second compressed data is recorded in the reverse direction from the last address of the recording area. For this reason, since the empty area is gathered in the intermediate portion sandwiched between the first compressed data and the second compressed data in the recording area, the recording efficiency of the compressed data in the buffer memory 15 can be further improved.

Furthermore, the electronic camera according to the present embodiment records the second compressed data in the recording area by inverting the order of arrangement. Therefore, if the second compressed data is read in the reverse direction from the tail address, the CPU 24 can read the second compressed data from the buffer memory 15 in the correct arrangement order.
(Supplementary items of the embodiment)
(1) The compression processing in the present invention is not limited to the time of imaging as in the above embodiment. For example, in the above embodiment, when the RAW data imaged in the second recording mode is digitally developed, the first compression processing unit 17 and the second compression processing unit 18 execute the compression processing in parallel. Good. Furthermore, the compression processing of the present invention may be realized in software by causing a separate processor core to function as two compression processing units in a computer having a plurality of processor cores.

(2) The image compression method in the present invention is not limited to DCT conversion conforming to the JPEG standard, and may be another known image compression method. For example, the first compression processing unit 17 and the second compression processing unit 18 of the above embodiment may perform wavelet transform processing in parallel on the main image divided in units of tiles in accordance with the JPEG2000 standard. Good.
(3) The method of dividing the main image in the present invention is not limited to the case where the image is divided into two in the horizontal direction and can be freely changed. For example, the image may be divided into two in the vertical direction. When the first compression processing unit 17 and the second compression processing unit 18 perform compression processing in accordance with the JPEG2000 standard, ROI (Region Of Interest) area compression processing is assigned to the first compression processing unit 17. The second compression processing unit 18 may be assigned compression processing for other areas.

  The present invention can be implemented in various other forms without departing from the spirit or the main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The block diagram which shows the structure of the electronic camera of this embodiment Flowchart for explaining compression processing in the electronic camera in the present embodiment The figure which shows the reading method of the image data typically The figure which shows the recording method of compressed data typically

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging optical system, 12 ... Image sensor, 13 ... Analog signal processing part, 14 ... A / D conversion part, 15 ... Buffer memory, 16 ... Image processing part, 17 ... 1st compression processing part, 18 ... 2nd compression Processing unit, 19 ... DMA control unit, 24 ... CPU

Claims (6)

A first compression processing unit that compresses input image data to generate first compressed data;
A second compression processing unit that compresses input image data to generate second compressed data;
A control unit that divides the image of one frame into two regions, and causes the first compression processing unit and the second compression processing unit to process the data of the divided images in parallel,
An image processing apparatus comprising: The image processing apparatus according to claim 1.
A memory for temporarily recording data,
A data transfer unit for recording the first compressed data and the second compressed data in a common recording area allocated for one frame on the memory;
An image processing apparatus further comprising: The image processing apparatus according to claim 2,
The data transfer unit records the first compressed data in the forward direction from the start address of the recording area, and records the second compressed data in the reverse direction from the end address of the recording area. Image processing device. The image processing apparatus according to claim 3.
The data transfer unit, wherein the arrangement order of the second compressed data is reversed and recorded in the recording area. The image processing apparatus according to claim 1.
The image processing apparatus, wherein the control unit calculates a parameter at the time of recompression based on a compression result of the first compression processing unit and the second compression processing unit. An imaging unit that captures a subject image and generates image data;
The image processing apparatus according to any one of claims 1 to 5,
An imaging apparatus comprising:

Images