JP2007006017A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus including a plurality of register groups for holding processing contents of image processing that can change the processing contents without intervention of a control unit. <P>SOLUTION: An image processing section 107A has respective processing blocks of a gamma correction section 201, a luminance color difference signal generating section 203, and a resolution conversion section 205, a counter 401 for counting an inputted processing unit, a register control section 402 for generating a signal S402 for selecting a register group in use depending on the numeral value of the counter 401, a storage section 403 that stores the order of inputted multiple processing units and the number of the register groups corresponding to the order, and register sections 404, 405, 406, wherein the respective register sections (404 to 406) include the plurality of register groups 407, 408, 409, respectively, and selectors 410 that select them and output. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that processes an image by a solid-state image sensor, for example.

  In general, an image processing apparatus that processes an image from a solid-state imaging device performs processing such as signal amplification, color space conversion, γ correction, contour correction, noise reduction, and resolution conversion in order to improve image quality.

In an image processing apparatus, when image processing is performed on a plurality of image data having different properties, such as still images, moving images, and images of different sizes, it is necessary to change the content and extent of image processing each time the image to be processed changes. .
In order to change the content and level of processing according to the image to be easily processed, different image processing units may be prepared for each processing, but there is a disadvantage that the hardware scale becomes large.

Therefore, even if the content and degree of image processing are different, the processing content is basically the same in many cases. Therefore, the processing is performed by a single image processing apparatus, and has a register group that holds the processing content and degree. A general method is to determine the contents and degree of image processing by referring to the contents of the register group during processing.
The contents of the register group are rewritten by a control device such as a microprocessor in accordance with the image to be processed.

In addition, it is necessary to switch the processing content of an image in units smaller than one frame, such as a device such as a hybrid camera described in Patent Document 1 that needs to handle a still image and a moving image at the same time.
JP 2004-96354 A

  However, if the number of register groups to be rewritten is large or the frequency is high, the microprocessor must be operated at high speed in order to obtain sufficient processing capability, but this is disadvantageous in terms of power consumption and hardware scale increase. Become.

  In the case of processing in units of frames, there is often no problem that interferes with other processing. However, for devices such as a hybrid camera described in Patent Document 1, it is necessary to handle still images and moving images simultaneously. When it is necessary to switch the processing contents of an image in units smaller than one frame, the register group is frequently set and the processing capability of the microprocessor as described above becomes a problem.

  It is an object of the present invention to provide an image processing apparatus that can change processing contents without intervention of a control device by having a plurality of register groups that hold processing contents of image processing.

  In addition, the present invention adds attribute information to input data and selects the processing content to be performed on the image data by the processing device itself according to the attribute information, thereby processing the image processing content in units smaller than one frame. It is an object of the present invention to provide an image processing apparatus capable of quickly switching processing with a single processing apparatus when it is necessary to change the process.

  An image processing apparatus according to a first aspect of the present invention includes a processing block that performs different processing for each processing unit, and a register unit that includes a plurality of register groups that hold image processing contents or / and setting of the processing block. Control means for selecting a register group to be used from among a plurality of register groups in the register unit and switching image processing contents and / or degree when performing processing.

  Preferably, the image input is divided into processing units smaller than one frame, and a register group to be used for each processing unit can be selected to perform image processing.

  Preferably, the control means generates a signal designating a register group for each processing unit.

  Preferably, the control means refers to the storage unit that holds the correspondence between the order of the input processing units and the register group to be used, and the content held in the storage means according to the input order of the processing units. And a control unit for obtaining a register group to be used.

  Preferably, the input image is input with an identifier added for each processing unit, and the control means refers to a storage unit that holds a correspondence between the identifier and a register group to be used, and contents held in the storage unit And a control unit for obtaining a register group to be used from the identifier added to the processing unit.

  Preferably, the input image is input with data specifying a register group to be used for each processing unit, and the control means determines a register group to be used by data specifying the register group added to the processing unit. A control unit to obtain.

  A second aspect of the present invention is an image processing apparatus that processes an input image from an image sensor, and includes an image processing unit that performs various processes on the input image from the image sensor. The processing unit includes a processing block that performs different processing for each processing unit, a register unit that includes a plurality of register groups that hold settings of image processing contents and / or degree of the processing block, and the register unit when performing processing. Control means for selecting a register group to be used from among the plurality of register groups and switching the content or / and degree of image processing.

  According to the present invention, the register unit of the processing block has as many register groups as the number of properties of the input image, so that not only still images and moving images but also images of multiple properties can be processed. Is done.

According to the present invention, when image processing with different properties such as still images and moving images is switched in units smaller than one frame, different γ correction, contour correction, and color for each processing unit without intervention of a control device such as a microprocessor. Image processing parameters such as correction and noise reduction can be set.
In addition, in order to process a plurality of resolution images within one frame, an input image is switched in units smaller than one frame, and an output image size setting parameter of the resolution converter is changed to obtain a plurality of resolution images. It becomes possible.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus that processes an input image from the solid-state imaging device according to the first embodiment.

  As shown in FIG. 1, the image processing apparatus 100 includes a solid-state imaging device 101 including a CCD or a CMOS sensor, a synchronization signal generation unit 102, a control unit 103, an image capture unit 104, a frame buffer unit 105, and an image input unit 106. An image processing unit 107, a storage device 108, a display unit 109, a microprocessor 110, and a processor bus 111.

  The solid-state imaging device 101 outputs the image data IM to the image capturing unit 104 in accordance with the synchronization signal S102 generated by the synchronization signal generation unit 102.

  The synchronization signal S102 generated by the synchronization signal generation unit 102 is also input to the control unit 103, and the control unit 103 outputs a control signal for controlling operation timing to each unit.

  The image capturing unit 104 receives the image data IM output from the solid-state imaging device 101 in accordance with a control signal from the control unit 103 and temporarily stores it in the frame buffer 105.

  The image input unit 106 reads out the temporarily stored image data S106 from the frame buffer 105 according to the control signal generated by the control unit 103 according to the synchronization signal generated by the synchronization signal generation unit 102, and sends it to the image processing unit 107. send.

The image processing unit 107 performs various image processing on the input image data S106, and an output signal S107 of the image processing is a display unit 109 for displaying the processed image, and a memory for storing the processed image. Sent to the device 108.
The configuration and function of the image processing unit 107 will be described in detail later.

  The microprocessor 110 can perform control such as register read / write of the synchronization signal generation unit 102, the control unit 103, the image capture unit 104, and the image processing unit 107 via the processor bus 111.

  Hereinafter, a specific configuration and function of the image processing unit 107 of this embodiment will be described in detail.

The image processing unit 107 according to the present embodiment has a plurality of register groups, holds a plurality of settings by having a plurality of register groups, and selects a register group to be used when performing processing. It is possible to switch the degree.
The image processing unit 107 divides the input image S106 into processing units finer than one frame, selects a register group to be used for each processing unit, and performs image processing.
The image processing unit 107 has a control unit that generates a signal for designating a register group for each processing unit as a function unit for selecting a register group to be used, and is input as a control function for selecting the register group to be used. A storage unit that holds the correspondence between the order of processing units and the register group to be used is obtained, and the register group to be used is obtained according to the input order of the processing units by referring to the contents held in the storage unit.

  First, a general configuration example of the image processing unit will be described, and then a preferable configuration example of the image processing unit will be described.

  FIG. 2 is a diagram illustrating a general configuration example of the image processing unit 107 in FIG. 1.

The image processing unit 107 performs processing such as signal amplification, color space conversion, γ correction, contour correction, noise reduction, and resolution conversion.
FIG. 2 represents an image processing unit that sequentially performs N types of processing in an abstract manner such as processing 1 (201), processing 2 (203), and processing N (205).
In FIG. 2, the function block of process 1 is represented as a γ correction unit 201, the function / block of process 2 as a luminance / color difference signal generation unit 203, and the function block of process 3 as a resolution converter 205.
Each process 1 to 3 has a register group (202, 204, 206), and changes the contents and degree of image processing according to the contents.
The register group 202 is provided corresponding to the γ correction unit 201, the register group 204 is provided corresponding to the luminance / color difference signal generation unit 203, and the register group 206 is provided corresponding to the resolution conversion unit 205.
The register groups 202, 204, and 206 are changed by a control device such as the microprocessor 110.

In the image processing apparatus 100, it is necessary to display and store a moving image at a constant frame rate.
In order to obtain a constant frame rate, it is necessary to update an image at regular intervals. Therefore, a series of processes from input to display / storage of the solid-state imaging device 101 must be performed within a predetermined time.
Here, the entire interval of one image is referred to as one frame period.

In general, there are some moving images and still images that have different image processing contents and levels, such as conversion coefficients from RGB to luminance color difference signals, γ correction, noise reduction, and resolution conversion.
For example, in the luminance / chrominance signal generation processing, conversion coefficients from RGB to luminance / chrominance signals, a gradation correction table in γ correction, a noise amount to be removed by noise reduction, an image size to be output by resolution conversion, and the like.

If still images are required during moving image processing, the image stored in the frame buffer unit 105 is read twice, the first time for moving image processing, the second time for still image processing, It is necessary to create images that have been processed separately.
At that time, when moving images are displayed in real time, it is necessary to process the moving images within one frame period because the display is not interrupted. Switching between still image and moving image processing in one frame unit In many cases, the processing is not in time.
In that case, by dividing the image into processing units smaller than one frame, it is possible to process a still image over a plurality of frame periods for each processing unit during the processing of a moving image.
The order of the processing is shown in FIG.

  FIG. 3 is a diagram illustrating a processing order of a still image over a plurality of frame periods for each processing unit during processing of a moving image.

In FIG. 3, image 1 (301) represents a still image, image 2 (302) represents a moving image at the same time, and image 3 (303) represents a moving image at the next time.
The frame is divided into processing units 304 smaller than one frame, the moving images of the image 2 (302) and the image 3 (303) are processed within one frame period, and the still image of the image 1 (301) is processed for each processing unit 304. Processing is performed over several frames.
By processing in this way, moving images can be processed in real time, and still images can be processed in a surplus time.
In this case, the setting for moving image is written in the register group (202, 204, 206) while the microprocessor 110 is processing the moving image, and the setting for still image is registered during the period of processing the still image. It is necessary to write to the group (202, 204, 206).
However, it is necessary to change the register contents in units smaller than one frame. The smaller the processing unit 304 is, and the larger the number of register groups (202, 204, 206) is, the more burden is placed on the microprocessor 110. The high processing capability of the microprocessor 110 is required.

  Therefore, in this embodiment, the configuration shown in FIG. 4 is adopted as a preferable configuration of the image processing unit 107.

FIG. 4 is a block diagram illustrating a preferred configuration example of the image processing unit of the present embodiment.
4, the same components as those in FIG. 2 are denoted by the same reference numerals for easy understanding.
In FIG. 4, similarly to FIG. 2, a γ correction unit 201 is represented as a function block of process 1, a luminance / color difference signal generation unit 203 is represented as a function block of process 2, and a resolution conversion unit 205 is represented as a function block of process 3.

In addition to the γ correction unit 201, the luminance / color difference signal generation unit 203, and the resolution conversion unit 205, the image processing unit 107A in FIG. 4 includes a counter 401 that counts input processing units, and a register group that is used from the numerical values of the counter 401. It has a register control unit 402 that generates a signal S402 to be selected, a storage unit 403 that holds the order of input multiprocessing units and the number of register groups corresponding thereto, and register units 404, 405, and 406.
The counter 401, the register control unit 402, and the storage unit 403 constitute a control unit.

Image data S106 is input for each processing unit 304 to the input T107 of the image processing unit 107A.
The image processing unit 107A includes a block for performing image processing from processing 1 (201) and processing 2 (203) to processing N (205). A register unit 404 for storing the settings of processing 1 (201), processing 2 There is a register unit 405 for saving the setting of (203) and a register unit 406 for holding the setting of process N (205).

Each register unit (404 to 406) includes a plurality of register groups 407, 408, and 409, and includes a selector 410 that selects and outputs them.
Although an example in which one register unit has three register groups is shown in this figure, the number of register groups may be increased or decreased depending on the type of processing.
The contents of each register group are set in advance by the microprocessor 110.

In the image processing unit 107A, the processing units 304 input by the counter 401 are counted, and the order of the input processing units 304 and the corresponding register group numbers are held in the storage unit 403.
Then, the register control unit 402 generates a signal S402 for selecting a register group to be used from the numerical value of the counter 401, and a number specifying the register group is transmitted from the register control unit 402 to each of the register units 404 to 406 as the signal S402. It is done.

  FIG. 5 is a diagram illustrating a data structure of the storage unit 403 in FIG.

As shown in FIG. 5, the storage unit 403 can hold a plurality of entries, with the number 502 of processing units to be input and a number 503 specifying a register group as one entry 501.
If the number of processing units 304 counted by the counter 401 is within the number of processing units of the first entry in the storage unit 403, the register group specified by the register group number of the first entry is used.
When the number of processing units counted by the counter 401 exceeds the number of processing units of the first entry in the storage unit 403, the next entry is referred to, and the number of processing units counted by the counter 401 is the number of processing units of the entry. The register group specified by the register group number of the entry is used for a number of times.
Similarly, when the reference is repeated until the last entry in the storage unit 403, the entry to be referred to may be returned to the top entry.

The image processing unit 107A counts the input processing unit 304 by the counter 401, and compares the counted number with the entry in the storage unit to register a register group number S402 used for processing the input processing unit image data. obtain.
The register group number S402 is transmitted to the register units 404, 405, and 406 of each process, and the register group to be used is selected by the selector 410.
With reference to the selected register group, processing 1 (201), processing 2 (203), and processing N (205) perform image processing on the input image data of the processing unit 304.

  As described above, according to the first embodiment, not only a still image and a moving image but also a plurality of properties are processed, the number of properties of the image input to the register unit of the processing block is the same. It is possible to process by having the register group.

FIG. 6 is a block diagram illustrating a configuration example of the image processing unit according to the second embodiment.
FIG. 7 is a diagram illustrating a data structure of a processing unit of image data input to the image processing unit in FIG.
In FIG. 6, the same components as those in FIG. 4 are denoted by the same reference numerals.
In FIG. 6, similarly to FIG. 2, a γ correction unit 201 is represented as a function block of process 1, a luminance / color difference signal generation unit 203 is represented as a function block of process 2, and a resolution conversion unit 205 is represented as a function block of process 3. Yes.

In the second embodiment, as shown in FIG. 7, an identifier indicating the type of image data is stored in the header portion 702, and image data is stored in the data portion 703.
In this embodiment, the image input unit 106 reads out image data from the frame buffer unit 105 for each processing unit 304, adds a header unit 702 to the image data 703, and sends it to the image processing unit 107 B as one processing unit 701.

  The image processing unit 107B according to this embodiment includes blocks that perform image processing from processing 1 (201) and processing 2 (203) to processing N (205), and a plurality of blocks for storing the settings of processing 1 (201). Register group 404 including register group 407 to 409 and selector 410, register unit 405 including register group 407 to 409 and selector 410 for saving setting of process 2 (203), setting of process N (205) A register unit 406 including a plurality of register groups 407 to 409 for holding and a selector 410 is provided.

In addition to the γ correction unit 201, the luminance / color difference signal generation unit 203, the resolution conversion unit 205, and the register units 404, 405, and 406, the image processing unit 107B in FIG. A header analysis unit 601 for reading an identifier added to the register, a register control unit 602 for generating a number for selecting a register group to be used from the identifier, and a number for specifying the identifier of the input processing unit 701 and the corresponding register group A storage unit 603.
The header analysis unit 601, the register control unit 602, and the storage unit 603 constitute a control unit.

  FIG. 8 is a diagram illustrating a data structure of the storage unit 603 in FIG.

  As shown in FIG. 8, the storage unit 603 includes a plurality of entries with a number 802 indicating the identifier of the header unit 702 added to the input processing unit 701 and a number 803 specifying a register group to be used as one entry. Can be held.

When the processing unit 701 is input to the image processing unit, the register control unit 602 searches for an identifier that matches the identifier read by the header analysis unit 601 and the identifier 802 of the entry held in the storage unit 603, The register group number of the entry that matches is sent to each of the register units 404, 405, and 406 as a signal S602.
The register group number S602 is transmitted to the register units 404, 405, and 406 of each process, and the register group to be used is selected by the selector 410.
With reference to the selected register group, processing 1 (201), processing 2 (203), and processing N (205) perform image processing on the input image data of the processing unit 701.

According to the second embodiment, the register group can be switched for each processing unit 701, and different image processing can be performed for each processing unit 701.
Even when processing an image having a plurality of properties as well as a still image and a moving image, processing can be performed by having as many register groups as the properties of the input image in the register unit of the processing block.

FIG. 9 is a block diagram illustrating a configuration example of an image processing unit according to the third embodiment.
FIG. 10 is a diagram illustrating a data structure of a processing unit of image data input to the image processing unit in FIG.
9, the same components as those in FIGS. 4 and 6 are denoted by the same reference numerals.
In FIG. 9, as in FIG. 2, the γ correction unit 201 is represented as the functional block of process 1, the luminance / color difference signal generation unit 203 is represented as the functional block of process 2, and the resolution conversion unit 205 is represented as the functional block of process 3. Yes.

In the third embodiment, as shown in FIG. 10, with the data structure of one processing unit 1001, register group numbers corresponding to the number of processing blocks performed on image data are added to the header unit 1002, and the header unit Subsequently, a data portion 1003 storing image data of 304 processing units follows.
In this case, there are N processing blocks.
In this embodiment, the image input unit 106 reads out image data from the frame buffer unit 105 for each processing unit 304, and sends a processing unit 1001 in which a header unit 1002 is added to the image data 1003 to the image processing unit 107C.

  The image processing unit 107 </ b> C of this embodiment includes a block that performs image processing from processing 1 (201) and processing 2 (203) to processing N (205), and a plurality of blocks for storing the settings of processing 1 (201). A register unit 404 including a register group 407 to 409 and a selector 410, a register unit 405 including a plurality of register groups 407 to 409 and a selector 410 for storing the setting of the process 2 (203), and a process N (205) setting. A register unit 406 including a plurality of register groups 407 to 409 for holding and a selector 410 is provided.

In addition to the γ correction unit 201, the luminance / color difference signal generation unit 203, the resolution conversion unit 205, and the register units 404, 405, and 406, the image processing unit 107C in FIG. A header reading unit 901 that reads the register group number added to the header, and a register control unit 902 that generates a signal S902 for selecting a register group from the register group number added to the header.
The header reading unit 901 and the register control unit 902 constitute control means.

According to the third embodiment, the image processing unit 107C performs processing by selecting a register group according to a register group number for each processing block added to the header unit 1002 of the input processing unit 1001, and performing processing. It is possible to switch the content and degree of image processing for each unit 1001.
In addition, since the register group number is added to the header portion, there is an advantage that the storage portion is unnecessary.

As described above, according to the present embodiment, when image processing having different properties such as a still image and a moving image is switched in units smaller than one frame, it is different for each processing unit without intervention of a control device such as a microprocessor. It is possible to set image processing parameters such as γ correction, contour correction, color correction, and noise reduction.
Further, in order to process a plurality of resolution images within one frame, an input image is switched in units smaller than one frame, and an output image size setting parameter of the resolution conversion device is changed to obtain a plurality of resolution images. It becomes possible.

It is a block diagram which shows the structural example of the image processing apparatus which processes the input image from the solid-state image sensor which concerns on 1st Embodiment. It is a figure which shows the general structural example of the image process part of FIG. It is a figure which shows the order of a process over several frame periods for every process unit during the process of a moving image. It is a block diagram which shows the suitable structural example of the image process part of this embodiment. It is a figure which shows the data structure of the memory | storage part of FIG. It is a block diagram which shows the structural example of the image process part which concerns on 2nd Embodiment. It is a figure which shows the data structure of the process unit of the image data input into the image process part of FIG. It is a figure which shows the data structure of the memory | storage part of FIG. It is a block diagram which shows the structural example of the image process part which concerns on 3rd Embodiment. It is a figure which shows the data structure of the process unit of the image data input into the image process part of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus, 101 ... Solid-state image sensor, 102 ... Synchronization signal generation part, 103 ... Control part, 104 ... Image capture part, 105 ... Frame buffer part, 106 Image input unit 107, 107A, 107B, 107C Image processing unit 108 Storage device 109 Display unit 110 Microprocessor 111 Processor bus 401 ... Counter, 402 ... Register controller, 403 ... Storage, 404 ... Register for processing 1, 405 ... Register for processing 2, 406 ... Register for processing N, 407: Register group 1, 408: Register group 2, 409: Register group 3, 410: Selector, 601: Header analysis unit, 602: Register control unit, 603,. Storage unit, 901 ... header reading unit, 902 ... register controller.

Claims (10)

A processing block that performs different processing for each processing unit;
A register unit having a plurality of register groups for holding image processing content or / and setting of the degree of the processing block;
An image processing apparatus comprising: a control unit that selects a register group to be used from among a plurality of register groups in the register unit and switches image processing content and / or degree when performing processing. The image processing apparatus according to claim 1, wherein the image input can be divided into processing units smaller than one frame and image processing can be performed by selecting a register group to be used for each processing unit. The image processing apparatus according to claim 1, wherein the control unit generates a signal designating a register group for each processing unit. The control means includes
A storage unit that holds the correspondence between the order of input processing units and the register group to be used;
The image processing apparatus according to claim 3, further comprising: a control unit that obtains a register group to be used in accordance with an input order of processing units with reference to contents stored in the storage unit. The input image is input by adding an identifier for each processing unit,
The control means includes
A storage unit that holds the correspondence between the identifier and the register group to be used;
The image processing apparatus according to claim 3, further comprising: a control unit that obtains a register group to be used from an identifier added to a processing unit with reference to contents stored in the storage unit. The input image is input with data specifying a register group to be used for each processing unit,
The control means includes
The image processing apparatus according to claim 3, further comprising a control unit that obtains a register group to be used by data specifying the register group added to the processing unit. An image processing apparatus for processing an input image from an image sensor,
An image processing unit that performs various processes on an input image from the image sensor;
The image processing unit
A processing block that performs different processing for each processing unit;
A register unit having a plurality of register groups for holding image processing content or / and setting of the degree of the processing block;
An image processing apparatus comprising: a control unit that selects a register group to be used from among a plurality of register groups in the register unit and switches image processing content and / or degree when performing processing. The control means includes
A storage unit that holds the correspondence between the order of input processing units and the register group to be used;
A control unit that obtains a register group to be used in accordance with the input order of processing units with reference to the contents held in the storage unit and generates a signal designating the register group for each processing unit. The image processing apparatus described. An image input unit that adds an identifier for each processing unit to the input image and inputs the input image to the image processing unit,
The control means of the image processing unit is:
A storage unit that holds the correspondence between the identifier and the register group to be used;
The control part which obtains the register group to be used from the identifier added to the processing unit with reference to the content held in the storage unit, and generates a signal designating the register group for each processing unit. Image processing device. An image input unit for adding data for specifying a register group to be used for each processing unit to the input image,
The control means of the image processing unit is:
The image processing apparatus according to claim 7, further comprising: a control unit that obtains a register group to be used by data designating a register group added to a processing unit and generates a signal designating the register group for each processing unit.

Images