JP2018007083A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for more simple and efficient data adjustment, in an image processing apparatus performing data adjustment in a pre-stage processing section and a post-stage processing section.SOLUTION: For the image data from an image pick-up device 12, a front end processor 18 performs temporary black level adjustment by using pixels in the first region of an optical black (OB) region, and subsequently performs noise reduction (NR) processing. A back end processor 20 acquires the image data processed by the front end processor 18, performs ideal black level adjustment by using pixels in the second region of OB region, and then performs NR processing. Temporary black level adjustment is performed with lower accuracy than true black level adjustment.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus that performs image processing through a pre-stage processing circuit and a post-stage processing circuit.

  There is known a digital camera that performs preprocessing on image data from an image sensor in a pre-processing unit, and then transfers the post-processing to the post-processing unit. In addition, in a camera that uses an image sensor with a high pixel rate, the front engine, which is a front-end processing unit, reduces the pixel rate, and the back engine, which is a rear-end processing unit, receives image data at the reduced pixel rate and performs image processing. The structure which gives is proposed (refer patent document 1).

Japanese Patent No. 5906846

  However, until now, when image processing is to be performed by the preceding image processing unit, it is necessary to correct the black level. Therefore, the black level adjustment processing similar to that performed at the subsequent stage is also performed by the preceding image processing unit. There is a problem that the circuit scale for black level adjustment becomes large. Further, there is a problem in that the image processing unit in the subsequent stage can perform black level correction similar to the process in the previous stage and the circuit scale is doubled.

  An object of the present invention is to enable data adjustment more simply and efficiently in an image processing apparatus that performs data adjustment in a pre-processing unit and a post-processing unit.

  The image processing apparatus according to the present invention acquires the image data processed by the preceding image processing unit, the pre-stage image processing unit that performs the first adjustment and the predetermined signal processing on the input image data, and performs the second adjustment. And a post-stage image processing unit that performs predetermined signal processing, and the first adjustment and the second adjustment are performed by different methods.

  The adjustment is, for example, optical black adjustment or dark current amount adjustment. Further, it is preferable that the second adjustment is performed with higher accuracy than the first adjustment. Further, the area used for the first adjustment is preferably different from the area used for the second adjustment, and the area used for the first adjustment is preferably narrower than the area used for the second adjustment. .

  The image data is image data from the image sensor, and the areas used for the first and second adjustments include, for example, at least a part of an optical black (OB) region of the image sensor. Further, the predetermined signal processing in the pre-stage and post-stage image processing units includes, for example, either a gain adjustment process or a noise reduction process.

  The former image processing unit includes first adjustment level holding means for holding the first adjustment level, and the latter image processing unit acquires the first adjustment level from the first adjustment level holding means, and uses the first adjustment level. Then, the second level adjustment may be corrected.

  A digital camera according to the present invention includes the above-described image processing apparatus.

  ADVANTAGE OF THE INVENTION According to this invention, in the image processing apparatus which performs data adjustment in a front | former stage process part and a back | latter stage process part, it is possible to adjust data more simply and efficiently.

It is a block diagram which shows the structure of the camera which is one Embodiment of this invention. It is a block diagram which shows the electrical structure centering on a front | former stage process part (henceforth a front end processor) and a back | latter stage process part (henceforth a back end processor). It is a top view of the image sensor which shows the 1st field and the 2nd field used for OB detection.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a camera equipped with a signal processing apparatus according to an embodiment of the present invention.

  The camera 10 is a single-lens reflex digital camera equipped with an image sensor 12 such as a CMOS or CCD, for example, and a mirror 16 capable of up / down is disposed between the lens 14 and the image sensor 12 (FIG. 1). Shows the mirror down state). The subject image is taken in a mirror-up state, and the subject image formed through the lens 14 is formed on the image sensor 12 and acquired as an image signal. The image signal detected by the image sensor 12 is sent to a front-end processor (pre-stage image processing unit) 18 using an application specific integrated circuit (ASIC) or the like, and after predetermined signal processing such as NR is performed, It is sent to a back-end processor (post-stage image processing unit) 20 using a DSP or the like.

  On the other hand, the light incident on the lens 14 in the mirror-down state is reflected by the main mirror 16M of the mirror 16 and imaged on the screen 22. The subject image formed on the screen 22 is guided to the eyepiece lens 26 through the pentaprism 24 and the like, and the user can observe the subject image through the eyepiece lens 26. Further, the light passing through the pentaprism 24 is also guided to the AE sensor 28 and used for automatic exposure control. Further, the mirror 16 includes a secondary mirror 16S. In the mirror-down state, a part of the light transmitted through the main mirror 16M is reflected by the secondary mirror 16S and detected by the AF sensor 30, and used for autofocus control. Further, the camera 10 of the present embodiment includes a thermometer 13, and the temperature in the camera is input to the back-end processor 20 and monitored.

  FIG. 2 is a block diagram showing an electrical configuration centering on the front-end processor 18 and the back-end processor 20 of the present embodiment.

  As shown in FIG. 2, image data from the image sensor 12 is input to the front end processor 18 via an input sensor I / F (interface) 32. The image data is sent to the OB detector 34, and a temporary black level is calculated using the pixel value of a predetermined area (first area) of the OB area of the image sensor 12. The OB subtracting unit 36 subtracts the black level calculated by the OB detecting unit 34 from the image data, and the gain adjusting unit 38 performs gain adjustment. The first OB region is not detected and may be a fixed value. The defect correction unit 40 interpolates image data corresponding to a pixel that is a defective pixel, and the NR unit 42 performs noise reduction processing on the image data. The black level obtained from the OB area of the first area includes the error E. Since the gain adjustment and NR are performed in the state including the error, the error E is multiplied by a coefficient by processing, and as a result, the error E ′ is output. Will remain. The image data that has been subjected to the noise reduction process is output to the back-end processor 20 via the output sensor I / F 44 after an offset is added in consideration of the error E ′ by the offset unit 43.

  Data including an error E ′ is input to the back-end processor 20 through the input sensor I / F (interface) 52 and the image data from the output sensor I / F 44 of the front-end processor 18. The image data is sent to the OB detector 54, and the error is canceled by detecting the OB including the error E ′ using the pixel value of the predetermined area (second area) of the OB area of the image sensor 12. A more accurate black level is calculated. Here, the second OB region uses a different area with less noise than the first OB region, uses a wider range of OB regions, or uses a different method, so that the black color is more accurate than the previous stage. The level can be calculated. Note that a correction value that takes into account the difference in black level between the effective portion and the OB area or a correction value of the dark current generation amount may be added to the calculated value. Further, individual differences of sensors that occur in the OB area may be taken into account. The correction amount may be a correction amount that varies depending on temperature and exposure time. The calculated black level is subtracted from the image data in the OB subtractor 56, and gain adjustment is performed in the gain adjuster 58. The defect correction unit 60 interpolates image data corresponding to the defective pixel, and the NR unit 62 performs noise reduction processing on the image data. The image data that has been subjected to noise reduction processing is subjected to image processing by the image processing unit 64 and then output to the image memory 66.

  The front-end processor 18 is controlled based on a control signal from the back-end processor 20, and a signal related to control, a signal related to a state, and an interrupt are exchanged between the two via the communication ports 68 and 70. Communication data transmitted and received via the communication ports 68 and 70 is stored in the respective registers 69 and 71, and settings and the like in the processors 18 and 20 are determined based on these data.

FIG. 3 is a plan view of the image sensor 12 showing the difference between the first region used for OB detection of the front end processor 18 and the second region used for OB detection of the back end processor 20. In the approximate center of the image sensor 12, there is an effective pixel area A _E used for photographing the object, and an OB area A _OB is provided around it. The first region, for example, the effective pixel area A region A1 at the upper side of the _E is used, the second region, for example, left and right regions of the effective pixel area A _E or the effective pixel area A and below the region of _E,, The combined area or the entire OB area _AOB is used. FIG. 3 shows a case where the lower area A2 of the _OB area AOB is used as the second area.

The black level in the front end processor 18 is a simple black level, and the area of the first region A1 is narrower than the area of the second region A2 because of a simpler technique than the black level in the back end processor 20. In addition, when calculating the black level using the first area A _OB common to the front-end processor 18 and the back-end processor 20, the front-end processor 18 eliminates the need for a circuit that eliminates the influence of defective pixels. in the calculation of the 20 black level, as a method for performing a process for removing all the defective pixels from the calculation of the same in the first region a _OB, performs processing by a simple method than the front-end processor 18 back-end processor 20 . Further, in the calculation of the black level of the front end processor 18, some or all of the defective pixels may be included in the calculation of the black level. This may be performed, for example, by setting a threshold value of a pixel value as a defective pixel so as to be sweeter than that of the back-end processor 20. A fixed value given in advance can be used as the black level in the front-end processor 18.

  As described above, the black level in the front end processor 18 is calculated by a simpler process than the black level process in the subsequent stage, or a fixed value is used. For this reason, the black level in the previous stage has an error compared to the ideal black level. At this time, it is possible to correct the black level with an error to the ideal black level by adjusting the black level in the subsequent stage again by using a method that is wider than the previous stage or using a different method with a higher accuracy than the previous stage. Is possible.

  However, the OB output of the sensor may not be an ideal value due to the influence of temperature, exposure time, and the like. Therefore, the detected black level may need to be corrected by temperature, exposure time, or the like. Since the front-end processor 18 of the present embodiment is intended to detect a black level by a simple method, the detected black level is not corrected due to differences in various conditions such as temperature and exposure time. That is, the black level detected by the OB detection unit 34 is subtracted from the image data by the OB subtraction unit 36 to perform temporary black level adjustment, and gain processing (38) and NR processing (42) are performed on the image data. Do. At this time, the front-end processor 18 records the provisional black level (first black level) detected by the OB detection unit 34 in the register 69.

  Here, the OB detection value OBF that is not corrected by the temperature and the exposure time is a value in which the offset value OFS due to the influence of the temperature, the exposure time, and the like is included in the ideal black level OB, and OBF = OB + OFS. Further, since the OB detection unit 34 of the front end processor 18 uses a simple method, the OB detection value in the OB detection unit 34 includes an offset value OBE of a detection error. Therefore, the actual OB detection value OBF ′ detected by the OB detection unit 34 is OBF ′ = OBF + OBE = OB + OFS + OBE. When the OB subtraction unit 36 subtracts the OB detection value OBF ′ from the image data, only the detection error offset value OBE remains in the image data, and only the detection error offset value OBE remains in the image data transferred to the back-end processor 20. Propagates. The back-end processor 20 detects the offset value OBE of the detection error by performing OB detection (OB detection unit 54) on the image data including the offset value OBE of the detection error.

  On the other hand, in order to correct the OB output of the image sensor (sensor) 12 with the temperature and the exposure time, the OB output of the image sensor 12 is required. Therefore, the back-end processor 20 reads the OB detection value OBF ′ (first black level) previously stored in the register 69 of the front-end processor 18 from the front-end processor 18 through the communication ports 68 and 70. Then, the back-end processor 20 uses the detected detection error offset value OBE and the provisional black level (first black level) OBF ′ read from the front-end processor 18 based on the temperature of the image sensor 12 and the exposure time. The black level OBF including the generated offset value OFS, that is, OBF = OBF′−OBE is calculated.

  The back-end processor 20 calculates the black level offset value OFS of the image sensor 12 from the temperature detected by the thermometer 13 and the exposure time, and corrects the previously calculated black level OBF to correct the image sensor 12. An ideal black level OB (= OBF-OFS) is calculated. The back-end processor 20 subtracts the calculated ideal black level OB from the image data by the OB subtraction unit 56 and performs gain adjustment by the gain adjustment unit 58. The defect correction unit 60 interpolates image data corresponding to the defective pixel, and the NR unit 62 performs noise reduction processing on the image data. The image data that has been subjected to noise reduction processing is subjected to image processing by the image processing unit 64 and then output to the image memory 66. Here, although an example of correction is expressed as an offset amount, the correction may be a method of correcting by multiplication or the like.

  As described above, according to the present embodiment, for example, the black level first adjustment is performed on the image data input by the front-end processor to perform predetermined signal processing, and then the back-end processor is different from the first adjustment. For example, in the image processing apparatus that performs data adjustment in the front-end processor and the back-end processor, it is possible to more simply and efficiently perform the data processing. Can be adjusted.

  For example, in the front-end processor, the number of pixels used for black level calculation can be greatly reduced compared to the back-end processor, and the calculation is simplified, so black level calculation, necessary registers, counters, memory, etc. Can be reduced, the circuit configuration can be simplified, and the same effect as that obtained by using the equivalent black level correction circuit in the front end portion and the back end portion can be obtained.

  In the present embodiment, the area used for the first black level adjustment is different from the area used for the second black level adjustment. For example, the area used for the first black level adjustment is changed to the second black level adjustment. By making the area smaller than the area used for the first processing, it is simpler than the subsequent processing unit, and the memory size, the register size, and the usage amount of registers such as a counter in the previous processing unit can be reduced.

  In the present embodiment, the circuit size of the first black level adjustment is reduced by making the method used for the first black level adjustment different from the method used for the second black level adjustment, and the method is simple. Black level adjustment can be performed. In addition, the circuit scale can be reduced by performing the first black level adjustment by using a specified fixed value without acquiring it from the image.

  Further, in the present embodiment, the preceding image processing unit includes first black level holding means for holding the first black level, and the subsequent image processing unit acquires the first black level from the first black level holding means, By correcting the second black level adjustment using the black level, it is possible to adjust the black level with high accuracy in the subsequent image processing unit.

  In this embodiment, a single-lens reflex digital camera has been described as an example. However, the present embodiment can also be applied to a mirrorless digital camera, a compact camera, or other electronic devices having a camera function. . In the present embodiment, the front-end processor performs NR processing on image data, but other processing may be performed.

10 Digital Camera 12 Image Sensor 14 Lens 16 Mirror 18 Front End Processor (Pre-stage Image Processing Unit)
20 Back-end processor (second-stage image processing unit)
32, 52 Input sensor I / F
34, 54 Optical black (OB) detection unit 36, 56 Optical black (OB) subtraction unit 40, 60 Defect correction unit 42, 62 Noise reduction (NR) unit 43 Offset unit 44 Sensor I / F for output

Claims (10)

A pre-stage image processing unit that performs a first adjustment on input image data and performs predetermined signal processing;
A post-stage image processing unit that acquires image data processed by the pre-stage image processing unit, performs second adjustment, and performs predetermined signal processing;
The image processing apparatus, wherein the first adjustment and the second adjustment are performed by different methods.   The image processing apparatus according to claim 1, wherein the adjustment is an optical black adjustment.   The image processing apparatus according to claim 1, wherein the adjustment is adjustment of a dark current amount.   The image processing apparatus according to claim 1, wherein the second adjustment is performed with higher accuracy than the first adjustment.   The image processing apparatus according to claim 1, wherein an area used for the first adjustment is different from an area used for the second adjustment.   The image processing apparatus according to claim 1, wherein an area used for the first adjustment is narrower than an area used for the second adjustment.   The image data is image data from an image sensor, and an area used for the first and second adjustments includes at least a part of an optical black (OB) region of the image sensor. The image processing apparatus as described in any one of -6.   The image processing apparatus according to claim 1, wherein the predetermined signal processing in the front-stage and rear-stage image processing units includes either gain adjustment processing or noise reduction processing.   The preceding image processing unit includes first adjustment level holding means for holding the first adjustment level, and the subsequent image processing unit acquires the first adjustment level from the first adjustment level holding means, and The image processing apparatus according to claim 1, wherein the second adjustment level is corrected using an adjustment level.   An image pickup apparatus equipped with the image processing apparatus according to claim 1.

Images