JP4802482B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method suitable for use in a digital camera.

  2. Description of the Related Art Conventionally, for example, in a digital camera that records digital image data acquired by photographing on a recording medium, multiple stages of image processing are performed in the process of finally obtaining image data to be recorded. For example, Bayer interpolation processing for converting Bayer data temporarily stored in a memory at the time of photographing, that is, image data including pixels of color information corresponding to a color filter of a primary color Bayer array into YUV data including RGB data for each pixel, after conversion Image processing such as scaling processing is performed to increase or decrease the number of pixels of the YUV data. In such image processing, the entire image data (Bayer data or YUV data) is not used as a processing unit, but is called a belt-like belt in which the entire screen A is divided in the vertical direction as shown in FIG. Data for a plurality of lines is used as a processing unit, and data is sequentially read from a working memory for each belt and processed. As shown in FIG. 5B, this belt may overlap with the previous belt depending on the processing content.

When the belt is used as a processing unit, belt information such as the size (number of lines) and position of the belt to be processed is required in each processing. Therefore, for example, in the image processing such as the Bayer interpolation processing and scaling processing described above, as shown in FIG. 6, for each belt, reading of belt data from the memory (DMR), Bayer interpolation processing (interpolation output), scaling processing is performed. Prior to a plurality of stages of data processing such as processing (magnification output) and data writing to memory (DMAW), the above-described belt information generation operation has been performed.
JP 2003-18610 A

  However, in the image processing using the belt as a processing unit as described above, the next belt information is generated after one belt processing is completed, and the next belt processing is started. Therefore, even if individual belt processing such as data reading (DMAR), interpolation output, scaling output, and data writing (DMAW) is performed by individual hardware resources, the image processing speed can be increased. There was a waste of waiting for processing. For example, a resource for reading data waits for processing from the end of data reading until the end of data writing. For this reason, there is a problem that hardware resources cannot be effectively used and there is a limit to increasing the image processing speed.

  The present invention has been made in view of such a conventional problem, and provides an image processing apparatus and an image processing method capable of increasing the processing speed when image processing is performed by dividing image data into a plurality of processing units. The purpose is to provide.

In order to solve the above-described problem, in the first aspect of the present invention, a plurality of stages of data processing are repeatedly executed for each processing target while sequentially switching image data of a predetermined processing unit to be processed in all image data. An image processing apparatus for performing a desired image processing, a plurality of stages of data processing means for starting operation in a predetermined order and continuously performing the plurality of stages of data processing, and the plurality of stages of data processing means Generating means for generating different specific information for each of the processing units required in common for each stage of data processing, and a plurality of stages of data processing means for holding the specific information generated by the generating means and a holding means provided in each of the data processing means of the plurality of stages, have a row data processing based on the specific information held in said holding means, said plurality of stages Each of the data processing means, the specific information held in the holding means at the start of each operation, and shall be updated to a particular information processing unit to be processed next generated by said generating means.

  In such a configuration, while a plurality of stages of data processing is repeatedly performed by a plurality of stages of data processing means, different specific information is generated for each processing unit in synchronization with the repeated operation, and the plurality of stages of data processing is performed. Since each of the means is individually held and a plurality of stages of data processing means perform data processing based thereon, the waiting time for the operations of the plurality of stages of data processing means during the repeated execution of the plurality of stages of data processing is eliminated. Can do.

According to a second aspect of the present invention, in response to the start of the operation of the last stage data processing means in the plurality of stages of data processing means, the generation means specifies the processing unit to be processed next. Was supposed to be generated.

  In such a configuration, specific information necessary for the next data processing is generated in parallel with the repeated execution of the data processing in a plurality of stages. Since not only the waiting time for the operation of the data processing means in the stage but also the waiting time for generating the specific information can be eliminated, the data processing in each stage can be repeated without interruption during the image processing.

According to a third aspect of the present invention, in response to the end of the operation of the foremost data processing means in the plurality of stages of data processing means, the processing to be processed next for the foremost data processing means. Control means for starting unit data processing is provided.
In order to solve the above-mentioned problem, in the invention of claim 4, a plurality of stages of data processing are repeated for each processing target while sequentially switching the image data of a predetermined processing unit to be processed in all the image data. An image processing apparatus that performs target image processing by executing a plurality of stages of data processing means that starts an operation in a predetermined order and continuously performs the plurality of stages of data processing, and the plurality of stages of data Generating means for generating different specific information for each of the processing units required in common for each stage of data processing by the processing means, and the multi-stage data processing for holding the specific information generated by the generating means Holding means provided in each of the means, and the plurality of stages of data processing means perform data processing based on the specific information held in the holding means, and Forming means, responsive to the start of operation of the data processing means of the final stage in the data processing unit of the plurality of stages, and those for generating specific information of the processing unit to be processed next.

In order to solve the above problem, in the invention of claim 5, a plurality of stages of data processing are repeated for each processing target while sequentially switching the image data of a predetermined processing unit to be processed in all the image data. An image processing apparatus that performs target image processing by executing a plurality of stages of data processing means that starts an operation in a predetermined order and continuously performs the plurality of stages of data processing, and the plurality of stages of data Generating means for generating different specific information for each of the processing units required in common for each stage of data processing by the processing means, and the multi-stage data processing for holding the specific information generated by the generating means Holding means provided in each of the means, the plurality of stages of data processing means perform data processing based on the specific information held in the holding means, In response to the end of the operation of the first-stage data processing means in the plurality of stages of data processing means, the first-stage data processing means is provided with a control means for starting data processing of the next processing unit. It was.
In the invention of claim 6 , the image data to be processed by the plurality of stages of data processing means includes at least one of Bayer data acquired by the image sensor or YUV data generated from the Bayer data. Any one was included.

Further, in the invention of claim 7, wherein the by the data processing plural stages of data processing means, a pixel interpolation processing for the Bayer data obtained by the imaging device, scaling the YUV data, and image data transfer processing It was assumed that at least one of them was included.

In the invention according to claim 8 , the predetermined processing unit in all the image data is image data for a plurality of lines, and the generation means includes information including the number of lines of the image data for the plurality of lines. The specific information is generated.

According to a ninth aspect of the present invention, there is provided storage means for storing the entire image data, and the plurality of stages of data processing repeatedly executed by the plurality of stages of data processing means includes the predetermined processing unit. It is assumed that at least one of reading processing of image data from the storage unit and writing processing to the storage unit is included.

According to the invention of claim 10, the object is achieved by repeatedly executing data processing in a plurality of stages for each processing target while sequentially switching the image data of a predetermined processing unit to be processed in all the image data. An image processing method for performing image processing, wherein a plurality of stages of data processing are sequentially performed in parallel, and in parallel with this process, each processing unit required in common for each stage of data processing The process of generating the specific information necessary for the next multi-stage data processing, and the specific information generated in this process is to perform new data processing at each stage at the start of the data processing operation at each stage. a step of holding separate purposes, and a step of performing data processing of the respective steps on the basis of the specific information held by this step, the step of the holding, the movement of data processing in each stage Specific information held at the beginning, and as a way to update the specific information generated by the step.

According to this method, it is possible to eliminate the waiting time for the operation of the plurality of stages of data processing means during the repeated execution of the plurality of stages of data processing.
According to the invention of claim 11, the object is achieved by repeatedly executing data processing in a plurality of stages for each processing target while sequentially switching the image data of a predetermined processing unit to be processed in all the image data. An image processing method for performing image processing, wherein a plurality of stages of data processing are sequentially performed in parallel, and in parallel with this process, each processing unit required in common for each stage of data processing The process of generating the specific information necessary for the next multi-stage data processing, and the specific information generated in this process is to perform new data processing at each stage at the start of the data processing operation at each stage. Including a step of individually holding as an object and a step of performing data processing at each step based on the specific information held at this step, and the step of generating includes the step of data processing at the plurality of steps. Kicking in response to start of operation of data processing in the final stage, and a method for generating specific information necessary for the data processing of the next several steps.
According to the invention of claim 12, the object is achieved by repeatedly executing data processing in a plurality of stages for each processing target while sequentially switching the image data of a predetermined processing unit to be processed in all the image data. An image processing method for performing image processing, wherein a plurality of stages of data processing are sequentially performed in parallel, and in parallel with this process, each processing unit required in common for each stage of data processing The process of generating the specific information necessary for the next multi-stage data processing, and the specific information generated in this process is to perform new data processing at each stage at the start of the data processing operation at each stage. The process of holding individually as a purpose, the process of performing data processing at each stage based on the specific information held at this process, and the operation of data processing at the front stage in the data processing at each stage In response to the end, and the method including the step of initiating the data processing of the processing unit to be processed next as the data processing at the first stage.

As described above, in the present invention , it is possible to eliminate the waiting time for the operation of the plurality of stages of data processing means during the repeated execution of the plurality of stages of data processing. Therefore, it is possible to perform image processing at a higher speed than conventional.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image processing apparatus 1 according to the present invention. This image processing apparatus 1 is incorporated in a digital camera, and is set with interpolation processing for generating YUV data from Bayer data output from an image sensor such as a CCD, and the number of pixels of the generated YUV data. This is for performing a scaling process that increases or decreases with a magnification, and has the following configuration. Note that the scaling process is a process for recording an image having a size smaller than the number of pixels of the image sensor, for example, and for enlarging the image using a digital zoom function of a digital camera.

  The image processing apparatus 1 includes a control unit 2 that controls the entire operation of the digital camera, a memory 3 such as a DRAM that temporarily stores Bayer data, YUV data, and the like, a DMAR 4, a Bayer interpolation block 5 that performs the interpolation process, It comprises a scaling block 6 for performing the scaling process and a DMAW 7. DMAR4 is a block for reading Bayer data from the memory 3 by DMA transfer, and DMAW7 is a block for writing YUV data after the change in the number of pixels to the memory 3 by DMA transfer.

  The control unit 2 includes a CPU, a program ROM, a RAM, a peripheral circuit (not shown) including an input / output interface, and the like, and each of the above-described steps is performed according to a procedure described later based on a program stored in the program ROM. Image processing using the belt (see FIG. 5) already described in blocks 4 to 7 as a processing unit, and operations associated therewith are performed.

  Each of the blocks 4 to 7 is provided with a belt information register 8 for holding next belt information a sent from the control unit 2 such as the size (number of lines) and position of the next belt to be processed. Each process is performed using the next belt information a held there. The DMAR 4, the Bayer interpolation block 5, and the scaling block 6 output a belt signal b to the next block and output image data c during processing. The DMAW 7 outputs a belt signal b to the control unit 2 and outputs image data c to the memory 3 during processing. At the same time, the activation signal d instead of the belt signal b is output from the control unit 2 to the DMAR 4, and the DMAR 4 also outputs the belt signal b to the control unit 2.

  The image data c is the aforementioned Bayer data and YUV data. The system from the memory 3 to the Bayer interpolation block 5 is Bayer data, and the other is YUV data. The belt signal b is a binary signal of “L” or “H”, which means that belt processing starts from “L” → “H”, and belt processing ends from “H” → “L” (start signal). The same applies to d). This also indicates the operating state of the block that outputs this signal. That is, the belt processing is “H” and the stop is “L”. The side that receives the belt signal b starts from “L” → “H”, and transitions to the end processing from “H” → “L”. The end process is to output all the data being processed remaining in the block and end the belt process.

  As a result, the individual operations of the blocks 4 to 7 are as follows. For example, taking the Bayer interpolation block 5 as an example, as shown in FIG. 2A, the belt information register 8 holds the next belt information a, and as shown in FIG. When the belt signal b from “H” becomes “H” (at the time of activation), the next belt information “a” is taken into the belt information register 8, and processing is performed according to the next belt information “a”.

  In the image processing apparatus 1, the control unit 2 executes the process shown in FIG. 3 at the time of shooting with the digital camera (at the time of image recording), so that each of the blocks 4 to 4 at the timing shown in FIG. Individual data processing according to 7 is performed. That is, the control unit 2 initializes the belt number i at the start of processing (step S1). Next, belt information (number of lines, position, etc.) of the first 0th belt (i-th belt) is calculated, and the calculated belt information, that is, the next belt information a is output to the belt information register 8 of each block 4-7. (Step S2). The belt information is calculated based on the above-described magnification in the scaling block 6, for example. Subsequently, it is confirmed whether or not the DMAR 4 is in a stopped state (step S3). At the beginning of the process, since DMAR4 is in a stopped state (YES in step S3), DMAR4 is activated by the activation signal d, and reading of the 0th belt data from the memory 3 is started (step S4).

  Thereafter, while the DMAW 7 that is the final block is in a stopped state, the process waits (NO in step S5). During this time, the Bayer interpolation block 5, the scaling block 6, and the DMAW 7 are started in order. Eventually, when DMAW 7 is activated, that is, when writing of data after image processing of the 0th belt is started (YES in step S5), the belt number is incremented and updated at that point (step S6). Thereafter, the process returns to step S2 (NO in step S7), the belt information of the updated belt number “1” is calculated, and the output to each of the blocks 4 to 7 is started as the next belt information a. Thereafter, when the DMAR 4 is stopped, that is, when the reading of the 0th belt data is completed (YES in step S3), the DMAR4 is started again to start reading the data of the first belt (step S4). When DMAW 7 is activated again and writing of data after image processing of the first belt is started (YES in step S5), the belt number is incremented and updated at that point (step S6).

  Thereafter, until the belt information of the last belt number is processed (output) as the next belt information a (NO in step S7), the process returns to step S2 and the above-described processing is repeated. In the meantime, in each of the blocks 5 to 7 except for the DMAW 4, at the time when the belt signal output from the preceding block becomes “H”, the control unit 2 automatically outputs the new next belt information a being output. Is stored in the belt information register 8, and the next belt information a held during the activation is used for processing. Thereby, in each block 4-7, the belt process after the 2nd belt is repeatedly performed at each starting timing (rise timing) shown in FIG. Then, after the processing of the belt information of the last belt number is completed (YES in step S7), the control unit 2 waits for the belt processing in the DMAW 7 to end (YES in step S8), and ends the processing for the image.

  As described above, according to the present embodiment, as shown in FIG. 4, belt data reading (DMR), Bayer interpolation processing (interpolation output), scaling processing (magnification output) from the memory 3 for each belt. ) And multiple stages of data processing such as data writing (DMAW) to the memory 3 can be repeated without interruption. Therefore, even if image processing such as interpolation processing and scaling processing is performed using a belt composed of image data for a plurality of lines as a processing unit, there is no waste in operation during processing, and image processing can be performed at higher speed than before. .

  In the present embodiment, the control unit 2 generates the next belt information a at the start timing of the DMAW 7. However, the generation timing of the next belt information a may be synchronized with the start timing of the DMAW 7. For example, it does not necessarily have to be the same time, and at least before the DMAR 4 is started for the next belt processing.

  In addition, the belt information register 8 is provided in each of the blocks 4 to 7, and each block 4 to 7 performs processing using the next belt information a held in the belt information register 8. The following may be used. That is, while the belt information register 8 in each block 4 to 7 is abolished, the control unit 2 may cause each block 4 to 7 to start outputting the next belt information a individually in accordance with each start timing. In addition, it may be made to continue to do so while the individual blocks 4-7 are activated. Even in this way, multiple stages of data processing can be repeated without interruption, and image processing such as interpolation processing and scaling processing can be performed even if a belt consisting of image data for multiple lines is used as a processing unit. Therefore, it is possible to perform image processing at a higher speed than in the past.

  In the present embodiment, the image processing apparatus 1 has been described in which a belt obtained by vertically dividing an image is a processing unit in image processing. However, the present invention is not limited to this, and the data processing unit is, for example, an image horizontally. It may be pixel data of the divided area. Further, it may be data of each image when image processing is performed across a plurality of images, as in the case where the image processing is composition processing. In this embodiment, the image processing apparatus incorporated in the digital camera has been described. However, the present invention is incorporated in other video equipment, an image recognition apparatus, or the like, or used alone. It can also be applied to. The types and contents of image data to be processed by the processing blocks for image processing and the number of processing blocks for image processing, that is, the number of processing stages (two stages in the present embodiment) are also described here. It is not limited to.

1 is a circuit diagram showing an outline of an image processing apparatus according to the present invention. It is explanatory drawing which shows the relationship between the content of a belt information register, and a belt signal. It is a flowchart which shows the processing content by a control part. It is a timing chart which shows the operation | movement procedure in image processing. It is a schematic diagram which shows the belt used as an image processing unit. It is a timing chart which shows the operation | movement procedure by the conventional image processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Control part 3 Memory 4 Data reading circuit (DMR)
5 Bayer interpolation circuit 6 Scaling circuit 7 Data reading circuit (DMAW)
8 Belt information register

Claims (12)

An image processing apparatus that performs target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit as a processing target in all image data,
A plurality of stages of data processing means for starting the operation in a determined order and continuously performing the plurality of stages of data processing;
Generating means for generating different specific information for each processing unit required in common for each stage of data processing by the plurality of stages of data processing means;
Holding means provided in each of the plurality of stages of data processing means for holding the specific information generated by the generating means,
Said plurality of stages of the data processing means, have line data processing based on the specific information held in said holding means,
Each of the plurality of stages of data processing means updates the specific information held in the holding means at the start of each operation to the specific information of the processing unit to be processed next generated by the generating means. An image processing apparatus. Said generating means, said plurality of stages in response to the start of operation of the data processing means of the final stage in the data processing unit, according to claim 1, wherein the generating the specific information of the processing unit to be processed next Image processing apparatus. In response to the end of the operation of the first-stage data processing means in the plurality of stages of data processing means, the first-stage data processing means includes a control means for starting data processing of a processing unit to be processed next. The image processing apparatus according to claim 1 , wherein the image processing apparatus is an image processing apparatus . An image processing apparatus that performs target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit as a processing target in all image data,
A plurality of stages of data processing means for starting the operation in a determined order and continuously performing the plurality of stages of data processing;
Generating means for generating different specific information for each processing unit required in common for each stage of data processing by the plurality of stages of data processing means;
Holding means provided in each of the plurality of stages of data processing means for holding the specific information generated by the generating means,
The plurality of stages of data processing means performs data processing based on the specific information held in the holding means,
The image processing apparatus is characterized in that the generating means generates specific information of a processing unit to be processed next in response to the start of operation of the data processing means at the final stage in the data processing means at the plurality of stages. An image processing apparatus that performs target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit as a processing target in all image data,
A plurality of stages of data processing means for starting the operation in a determined order and continuously performing the plurality of stages of data processing;
Generating means for generating different specific information for each processing unit required in common for each stage of data processing by the plurality of stages of data processing means;
Holding means provided in each of the plurality of stages of data processing means for holding the specific information generated by the generating means,
The plurality of stages of data processing means performs data processing based on the specific information held in the holding means,
Further, in response to the end of the operation of the first stage data processing means in the plurality of stages of data processing means, a control means for causing the first stage data processing means to start data processing for the next processing unit. An image processing apparatus comprising the image processing apparatus. The image data to be processed by the plurality of stages of data processing means includes at least one of Bayer data acquired by an image sensor or YUV data generated from the Bayer data. Item 6. The image processing device according to any one of Items 1 to 5 . The data processing by the plurality of stages of data processing means includes at least one of pixel interpolation processing, YUV data scaling processing, and image data transfer processing for Bayer data acquired by the image sensor. the image processing apparatus according to any one of claims 1 to 5, characterized. The predetermined processing unit in all the image data is image data for a plurality of lines,
It said generating means, the image processing apparatus according to information including the number of line image data of the plurality of lines to one of claims 1 to 7, characterized in that generated as the specific information. Storage means for storing the entire image data;
The multi-stage data processing repeatedly executed by the multi-stage data processing means includes at least one of reading processing from the storage means and writing processing to the storage means of the image data of the predetermined processing unit. the image processing apparatus according to any one of claims 1 to 8, characterized in that one of or contained. An image processing method for performing target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit to be processed in all image data,
A process of sequentially performing multiple stages of data processing in sequence,
In parallel with this step, a step of generating specific information necessary for the next plurality of stages of data processing, which is different for each processing unit required in common for each stage of data processing,
Specific information generated in this process, the process of holding individually for the purpose of performing new data processing of each stage at the start of operation of data processing of each stage ;
And a step of performing data processing at each stage based on the specific information held in this step,
In the image processing method , the holding step updates the specific information held at the start of the data processing operation in each step to the specific information generated in the step . An image processing method for performing target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit to be processed in all image data,
A process of sequentially performing multiple stages of data processing in sequence,
In parallel with this step, a step of generating specific information necessary for the next plurality of stages of data processing, which is different for each processing unit required in common for each stage of data processing,
Specific information generated in this process, the process of holding individually for the purpose of performing new data processing of each stage at the start of operation of data processing of each stage;
And a step of performing data processing at each stage based on the specific information held in this step,
In the image processing method, the generating step generates specific information necessary for the next plurality of stages of data processing in response to the start of the operation of the last stage of data processing in the plurality of stages of data processing. An image processing method for performing target image processing by repeatedly executing data processing of a plurality of stages for each processing target while sequentially switching image data of a predetermined processing unit to be processed in all image data,
A process of sequentially performing multiple stages of data processing in sequence,
In parallel with this step, a step of generating specific information necessary for the next plurality of stages of data processing, which is different for each processing unit required in common for each stage of data processing,
Specific information generated in this process, the process of holding individually for the purpose of performing new data processing of each stage at the start of operation of data processing of each stage;
A step of performing data processing at each stage based on the specific information held in this step;
In response to the end of the data processing operation at the first stage in the data processing at each stage, starting the data processing of the processing unit to be processed next as the first data processing;
An image processing method comprising:

Images