JP4179777B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus used in an electronic imaging apparatus such as a digital camera.
[0002]
[Prior art]
In recent years, the number of pixels of an image pickup element used in an image pickup apparatus such as a digital camera has increased dramatically, and the amount of image data handled has become very large. For this reason, the processing time for handling image data has become a problem, and a high-speed image processing apparatus capable of reducing the processing time is required.
[0003]
A technique described in Japanese Patent Application Laid-Open No. 2000-31327 is known as a technique capable of speeding up such image data processing.
[0004]
FIG. 9 is a block diagram showing a configuration of a conventional image processing apparatus.
[0005]
In this image processing apparatus, a signal obtained by processing the image pickup signal from the CCD image pickup device by the preprocess circuit unit 3 under the control of the CPU 2 connected to the bus 1 is input to the image process processing unit 5 via the bus 1. The The image process processing unit 5 performs image processing by a pipeline system connected in series from the first-stage image process circuit unit 5-1 to the n-th image process circuit unit 5-n, and sends the result to the JPEG processing unit 6. The JPEG processing unit 6 is configured to record data relating to the compressed image formation in the frame memory 4 or a memory card (not shown) via the bus 1 after performing compression processing by the JPEG method.
[0006]
With this configuration, data transfer via the bus 1 is processed only by transfer from the frame memory 4 to the first-stage image process circuit unit 5-1 and transfer from the JPEG processing unit 6 to the frame memory 4. Therefore, the data transfer amount is considerably reduced as compared with the conventional method in which data transfer is repeatedly performed between the frame memory 4 and each image process circuit.
[0007]
[Problems to be solved by the invention]
However, there is still a strong demand for speeding up image data processing using this image processing apparatus to further reduce the processing time. And when trying to solve this problem, there are conditions to be considered.
[0008]
In the JPEG method, which is a standard compression method, processing such as compression is performed in units of blocks called MCUs (Minimum Coded Units), and the size of this MCU is usually an 8 × 8 block or a block that is an integer multiple of 8 It is done.
[0009]
For this reason, when image data is transferred from the image process processing unit 5 to the JPEG processing unit 6, it is necessary to divide the image data into MCU blocks of a predetermined size and transfer them to the JPEG processing unit 6. In addition to this condition, in order to be able to restore the data compressed by the JPEG processing unit 6 as the original image data, the image process processing unit 5 converts the divided image data into a JPEG processing unit in a predetermined order. 6 must be handed over.
[0010]
Therefore, when trying to shorten the image data processing time in this image processing apparatus, the measure must be in a form that satisfies the above-described constraints and can ensure the operation of the JPEG processing unit 6, and preferably There is a need for measures in a form that does not change the operation of the JPEG processing unit 6.
[0011]
The present invention has been made in view of such circumstances, and it is an object of the present invention to provide an image processing apparatus that can reduce the processing time of image data and does not change the configuration of the JPEG processing unit 6.
[0012]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides an image processing unit connected in series for processing supplied image data, and an image compression process for compressing data relating to image formation obtained by the image processing unit. An image processing apparatus including a memory for storing image data, and a basic data size when the image compression processing unit performs compression processing when generating data to be used other than image display by the image compression processing unit Image data having a vertical data size obtained by multiplying the data size in the vertical direction by an integer (a natural number of 2 or more) with respect to the size of a certain MCU and adding peripheral data to the memory successively in the horizontal direction from the memory and data supply means for supplying to the means, when supplied to the image compression processing unit to generate the data to be used in addition to the image display, the image The image compression processing means for dividing the data related to image formation obtained by the processing means into the basic data size and generating data used for image display from the data of the basic data size obtained by the division And an order changing means for supplying the image compression processing means to the image compression processing means in an order different from the case of supplying to the image processing apparatus.
[0013]
According to the present invention, in the image processing apparatus according to the above-described invention, processing operations of the image processing unit and the order changing unit are set so that processing corresponding to the data size supplied by the data supply unit can be performed. An image processing apparatus further comprising setting means .
[0014]
According to the present invention, in the image processing apparatus according to the above-described invention, the image compression processing means compresses data to generate data to be used for the display using a compression result of data to be used other than the generated image display. The order changing means is an image processing apparatus that supplies data divided into the basic data size to the image compression processing means in different orders in the initial compression process and the subsequent compression process .
[0015]
According to the present invention, in the image processing apparatus according to the above-described invention, the image data is obtained by imaging, and the image compression processing means compresses the data related to image formation by the JPEG method.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
In order to achieve the above-mentioned problem, the inventor repeatedly investigated the processing time delay factor in the image process processing unit 5 and continued to study persistently, and arrived at the present invention which is a countermeasure that does not affect the JPEG processing unit 6. It is a thing.
[0017]
First, one factor for delaying the processing time in the image process processing unit 5 will be described.
[0018]
FIG. 1 is a diagram showing the capacity of data processed in the image process processing unit 5.
[0019]
A data amount 11 indicates the capacity of image data read from the frame memory 4. This image data has a data capacity larger than the data amount 14 input to the JPEG processing unit 6 for the reason described later. Then, during the sequential data processing from the first-stage image process circuit unit 5-1 to the last-stage image process circuit unit 5-n of the image process processing unit 5, the capacity of the image data is the data amount 12,. Decrease.
[0020]
The reason why the data capacity is reduced in this way is that extra data is required to perform image processing. For example, when performing spatial filter processing in the image process processing unit 5, it is necessary to calculate using some data around the data to be processed, and the extra data is unnecessary after the filter processing. It is because it is removed as.
[0021]
FIG. 2 is a diagram for explaining an input / output state of image data in the image process processing unit 5.
[0022]
As described above, the input data of the image process processing unit 5 is an extra data required only for data processing in the capacity of 8 × 8n which is the data portion 15 composed of MCU blocks passed to the JPEG processing unit 6. An α × 8n capacity which is the portion 16 is added. Similarly to the data input to the image process processing unit 5 following this data, an extra data portion 18 is added to the data portion 17 composed of MCU blocks passed to the JPEG processing unit 6.
[0023]
Therefore, the data input to the image process processing unit 5 always includes redundant redundant data, and as the number of times that image data is read from the frame memory 4 increases, so much redundant data transfer occurs. It will be.
[0024]
FIG. 3 is a diagram showing the transfer order of the image data 20 according to the conventional method.
[0025]
(1) in FIG. 3 shows a state in which the image data 20 is divided into MCU blocks, and the order of the arrows shown in the figure, that is, MCU1, 2 in units of the size of (Y + α) pixels in the vertical direction. ,..., 5, 6,..., 9, 10,. Here, Y represents the vertical pixel size of one MCU block.
[0026]
(2) in FIG. 3 shows the order of blocks of image data flowing through the image process processing unit 5. The JPEG processing unit 6 sequentially receives and compresses the blocks input in the order indicated by the arrows, that is, the order of MCU1, 2, 3,..., And then transfers them to the frame memory 4.
[0027]
The inventor has paid attention to the extra data portion as a method for realizing higher-speed processing than the current processing configuration as described above.
[0028]
Here, the overlap amount L is defined by equation (1).
[0029]
L = α / (Y + α) (1)
This expression (1) is an index representing the ratio of time spent for processing extra data in the time to process data.
[0030]
Therefore, if the overlap amount L can be reduced, the wasteful data processing time can be reduced and the processing time of only necessary data can be approached, so that the processing speed can be increased.
[0031]
Here, α used for the overlap amount L is a value that is fixedly determined if the processing method used in the image process processing unit 5 is determined. For example, when LPF (low-pass filter) processing with 6 filter filter taps is performed, 5 is added to α, and LPF (low-pass filter) processing with 4 filter taps is performed. 3 is added to α. For this reason, it is possible to relatively reduce the overlap amount L by using an image process circuit having a small number of filter taps. However, this may not satisfy the processing function with the accuracy required originally. It cannot be adopted as a countermeasure.
[0032]
Therefore, Y, which is the number of pixels in the vertical direction, is set to a plurality of MCU sizes, i.e., an integral multiple of the MCU size, instead of the size of one MCU block as in the present situation, and the Y is increased to relatively overlap. A method of reducing the amount L is adopted.
[0033]
FIG. 4 is a diagram showing the transfer order of the image data 20 according to the method of the present invention.
[0034]
(1) in FIG. 4 shows a state in which the image data 20 is divided into a plurality of MCU blocks and has a size corresponding to the number of pixels of the plurality of MCUs in the vertical direction as compared with the conventional dividing method. The difference is that Y is set. This data is in the vertical direction in units of (Y + α) pixels. The order of the arrows shown in the figure, that is, (MCU1, 6, 11), (MCU2, 7, 12), (MCU3, 8, 13)...
[0035]
(2) in FIG. 4 shows the order of blocks of image data input to the image process processing unit 5, that is, (MCU1,6,11), (MCU2,7,12), (MCU3,8,13) as described above. )... (3) of FIG. 4 shows the order of the MCU blocks of the image data output from the image process processing unit 5, that is, the order of MCU1, 6, 11, 2, 7, 12, 3, 8, 13. ing. The JPEG processing unit 6 compresses the MCU blocks input in this order and transfers them to the frame memory 4.
[0036]
Note that the order of (3) in FIG. 4 is not limited to this embodiment, and is output from the process processing unit 5 in the order of MCU1, 6, 11, 12, 7, 2, 3, 8, 13. You may comprise.
[0037]
In this way, if a method of transferring a plurality of blocks is used, the overlap amount L can be reduced and the processing speed can be increased. However, in order to adopt this method, the following two points must be considered.
[0038]
The first point is that the capacity of data transferred to the image process processing unit 5 is increased as compared with the conventional case. In this regard, the configuration of the conventional image process processing unit 5 has an internal buffer with a capacity sufficient to cope with this increase in data capacity. Even if this method is adopted, the hardware is greatly increased. It can be processed with minor modifications without remodeling.
[0039]
The second point is that the order of transferred MCU blocks is changed. Since the order of assembling the images is changed by changing the transfer order, for example, when an image assembled according to the conventional order is displayed on the monitor, the image of the jigsaw puzzle pieces is not in an appropriate state. Will be displayed.
[0040]
In order to solve this problem, it is conceivable to change the data transfer order in the middle of the image process. However, the present invention aims to cope with this without changing the conventional hardware configuration. Consider whether the compression processing is possible even when the order of the data blocks transferred in the JPEG processing unit 6 is changed. Hereinafter, the compression operation by the so-called iterative approach method of the JPEG processing unit 6 in the normal state will be described.
[0041]
FIG. 5 is a flowchart showing a schematic operation of the JPEG processing unit 6.
[0042]
First, the JPEG processing unit 6 reads out the compression rate set by the operator of the image processing apparatus and sets it as the compression rate target value (S1). Subsequently, data compression for the first pass is performed under a compression processing parameter (for example, a scale factor) set to a predetermined initial value (S2). Then, the ratio between the data capacity of the compressed image and the data capacity of the original image is obtained, and the value of the second pass parameter for achieving the target compression rate is calculated based on the actual compression value (S3). .
[0043]
FIG. 6 is a diagram illustrating the relationship between the compression rate and the parameters.
[0044]
In this figure, the vertical axis represents the compression rate, and the horizontal axis represents the parameter value. Even with the same parameter value, the capacity to be compressed differs depending on the complexity of the image, and as shown in the figure, the characteristic curves 22, 23, and 24 have different characteristics depending on the complexity of the image.
[0045]
Therefore, using a plurality of characteristic curves obtained in advance through experiments or the like as a reference line, a characteristic curve 23 that matches the image is specified from the actual compression value in one pass and the initial parameter values. Then, a parameter correction value that gives the target compression rate is obtained according to the characteristic curve 23.
[0046]
Then, the data compression of the second pass is performed using the parameter correction value thus calculated (S4). When the target compression rate cannot be achieved by the second-pass compression (S5), fine adjustment of parameters by a predetermined algorithm is performed to determine a parameter value for the next pass (S6). Based on the above, data compression of the next pass is performed (S7). Thus, at least two compression processes are required to obtain a desired compression rate in the JPEG process.
[0047]
The inventor has found that in this JPEG processing operation, the image data processed in the first pass is used only for calculating the compression ratio under the initial parameter value, and is not used for other purposes such as display. Pay attention. If it does so, it will be concluded that even if it is the data with which the above-mentioned transfer order was changed, it can be used as data of the 1st pass.
[0048]
The present invention has been created on the basis of the above examination and consideration, and is configured as follows to achieve the above object.
[0049]
FIG. 7 is a block diagram showing the configuration of the image processing apparatus according to the embodiment of the present invention. In this figure, the same components as those in FIG. 9 are denoted by the same reference numerals, and detailed description thereof is omitted.
[0050]
In the present embodiment, in contrast to the configuration shown in FIG. 9, a selection unit 7 that is a means for selecting a method for transferring image data from the frame memory 4 to the image process processing unit 5, and JPEG processing from the image process processing unit 5. The difference is that an order changing unit 8 for changing the order of image data transferred to the unit 6 is further provided.
[0051]
A method for performing image processing at high speed in the image processing apparatus having this configuration will be described.
[0052]
FIG. 8 is a flowchart showing a schematic procedure of the image processing apparatus.
[0053]
When trying to compress an image, the selection unit 7 transfers the above-mentioned multiple MCU blocks to the image process processing unit 5 and the order change unit 8 based on a command from the CPU 2 (hereinafter referred to as “high speed”). An instruction to execute “the processing mode” and the parameter values necessary for executing the high-speed processing mode are set (S10).
[0054]
Examples of this parameter include the number of multiple MCU blocks to be transferred, the number of pixels to be transmitted, and the number of filter taps of the image process circuit.
[0055]
Then, the CPU 2 reads out image data in units of a plurality of MCU blocks from the frame memory 4 and transfers them to the image process processing unit 5 together with extra data required only for image data processing (S11).
[0056]
The image process circuit units 5-1,..., 5-n in each stage process the transferred image data of a plurality of MCU blocks in accordance with parameter values instructed in advance by the CPU 2 (S12). Then, the order changing unit 8 divides a plurality of MCU blocks in the image data image-processed by the image processing circuit unit 5-n at the final stage into a single MCU block, and sequentially sends it to the JPEG processing unit 6 in MCU units (S13). ).
[0057]
The JPEG processing unit 6 performs the first-pass compression process according to the above-described compression procedure (S14), and calculates a parameter value for the second-pass compression based on the compression result value obtained as a result (S15).
[0058]
Subsequently, the selection unit 7 performs a method (hereinafter referred to as “normal processing mode”) of sequentially transferring a single MCU block to the image process processing unit 5 and the order changing unit 8 based on a command from the CPU 2. And the parameter values necessary for executing the normal processing mode are set (S16).
[0059]
Then, the CPU 2 reads image data including a single MCU block from the frame memory 4 and transfers it to the image process processing unit 5 together with extra data required only for image data processing (S17).
[0060]
The image process circuit units 5-1,..., 5-n at each stage process the transferred image data according to the parameter values of the normal processing mode specified in advance, and the order change unit 8 performs the image process circuit at the final stage. The image data processed by the unit 5-n is sent to the JPEG processing unit 6 without changing the order of the image data (S18).
[0061]
The JPEG processing unit 6 performs the second and subsequent compression processes according to the above-described procedure (S19), and transfers the compressed data to the frame memory 4 (S20).
[0062]
In this way, by configuring the image process processing that is normally processed in units of one MCU to process a plurality of MCUs together, it is possible to further speed up image processing by a high-speed pipeline system. it can.
[0063]
Also, by applying the processing method configured in this way to the first pass of JPEG compression processing, the processing speed can be increased without significantly modifying the conventional hardware.
[0064]
In the embodiment, the present invention is described as applied to an imaging apparatus. However, the present invention is not limited to this example, and an image having a function of performing processing using extra data, such as spatial filtering processing. The present invention can be widely applied to processing apparatuses. Further, although the JPEG processing method has been described as an example of the compression processing, the present invention is not limited to this example, and can be widely applied to a compression method adopting an iterative approach method.
[0065]
【The invention's effect】
As described above, according to the present invention, the processing time of image data can be shortened without changing the configuration of the JPEG processing unit.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a volume of data processed in an image process processing unit.
FIG. 2 is a diagram illustrating an input / output state of image data in an image process processing unit.
FIG. 3 is a diagram illustrating a transfer order of image data according to a conventional method.
FIG. 4 is a diagram showing a transfer order of image data according to the method of the present invention.
FIG. 5 is a flowchart showing a schematic operation of a JPEG processing unit.
FIG. 6 is a diagram illustrating a relationship between a compression rate and a parameter.
FIG. 7 is a block diagram showing a configuration of an image processing apparatus according to the embodiment of the present invention.
FIG. 8 is a flowchart showing a schematic procedure of the image processing apparatus.
FIG. 9 is a block diagram showing a configuration of a conventional image processing apparatus.
[Explanation of symbols]
1 ... Bus 2 ... CPU
4 ... Frame memory 5 ... Image process processing unit 6 ... JPEG processing unit 14 ... Data amount 15 input to the JPEG processing unit 15 ... Data portion 16 composed of MCU blocks ... Data portion 20 required only for data processing ... Image data

Claims (4)

An image processing apparatus having image processing means connected in series for processing supplied image data and image compression processing means for compressing data relating to image formation obtained by the image processing means,
A memory for storing image data;
When generating data to be used for purposes other than image display by the image compression processing means, the data size in the vertical direction is set to an integer (two or more) with respect to the MCU size which is the basic data size when the image compression processing means performs compression processing A data supply means for supplying image data having a vertical data size obtained by adding peripheral data to a multiple of a natural number) continuously from the memory to the image processing means in the horizontal direction;
In the case of supplying to the image compression processing means to generate data used for other than the image display, the data relating to image formation obtained by the image processing means is divided into the basic data size and obtained by the division. Order changing means for supplying the data of the basic data size to the image compression processing means in a different order from the case of supplying the data to the image compression processing means for generating data used for image display ;
An image processing apparatus comprising: 2. The apparatus according to claim 1, further comprising setting means for setting processing operations of the image processing means and the order changing means so that processing corresponding to the data size supplied by the data supply means can be performed. Image processing apparatus. The image compression processing means performs a compression process to generate data used for the display using a compression result of data used other than the generated image display ,
The said order change means supplies the data divided | segmented into the said basic data size to the said image compression process means in a different order by the first compression process and the subsequent compression process . Image processing device. The image data is obtained by imaging,
The image processing apparatus according to any one of claims 1 to 3, wherein the image compression processing unit compresses data relating to image formation by a JPEG method.

Images