JP6115410B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  2. Description of the Related Art Conventionally, an imaging apparatus such as a digital still camera that can record digital data of a captured image has been widely used. The imaging device measures the intensity of light from the subject (photometry), and determines the exposure amount (time during which light is applied to the imaging element (shutter speed) and the intensity of the light (aperture value)). In the case of photographing conditions such as backlight, it is difficult to calculate an appropriate exposure amount. In the image obtained in such a case, the brightness of the whole or a specific part tends to be insufficient or excessive. For this reason, the image processing apparatus performs contrast correction on the image data (see, for example, Patent Documents 1 and 2).

JP 2003-46859 A JP 2008-124653 A

  However, tone correction (tone jump) may occur due to contrast correction, and image quality deteriorates.

  According to one aspect of the present invention, a first correction is performed by contrast-correcting the pixel data based on a gain value and an offset value corresponding to the luminance maximum value and luminance minimum value of imaging data of one screen including a plurality of pixel data. A pixel correction unit configured to generate pixel data, a second pixel data obtained by adding a predetermined value to the input pixel data, and a second pixel data obtained by performing the contrast correction; and a third correction unit that performs gamma correction on the first pixel data according to a correction table. According to a difference value between pixel data, a gamma correction unit that generates fourth pixel data obtained by performing gamma correction on the second pixel data according to the correction table, and the third pixel data. And a gradation correction unit for generating fifth pixel data obtained by correcting the third pixel data (P2a).

  According to one aspect of the present invention, it is possible to suppress deterioration in image quality.

It is a block diagram which shows the outline of an imaging device. (A) (b) is explanatory drawing of imaging data. It is a block diagram of an image processing part. (A) and (b) are explanatory diagrams of image processing. It is a block diagram of a gamma correction part. It is explanatory drawing of a gamma correction curve. It is explanatory drawing of contrast correction | amendment and (gamma) correction. It is a block diagram of a gradation correction unit. (A) to (c) are pixel value histograms.

Hereinafter, an embodiment will be described.
1 is a digital still camera (DSC), for example, and includes an imaging unit 10, an image processing unit (ISP: Image Signal Processor) 20, an operation unit 31, a memory (storage unit) 32, and a display device 33. ing.

  The imaging unit 10 outputs image data corresponding to incident light based on the subject. The image processing unit 20 performs various types of image processing on the image data output from the imaging unit 10 according to a predetermined set value or a setting by operating the operation unit 31. Then, the image processing unit 20 stores the image data being processed or after the processing in the memory 32. Further, the image processing unit 20 displays an image based on the image data output from the imaging unit 10 or an image based on the image data stored in the memory 32 on the display device 33. Then, the image processing unit 20 stores the image data stored in the memory 32 in the memory card 35 according to the operation of the operation unit 31, for example.

The imaging unit 10 includes an imaging optical system 11 and an imaging element unit 12.
The imaging optical system 11 includes a lens (such as a focus lens) that collects light from a subject, a diaphragm that adjusts the amount of light that has passed through the lens, and the like, and guides an optical subject image to the imaging element unit 12. The image sensor unit 12 includes, for example, a Bayer array color filter and an image sensor. The imaging device is a CCD (Charge Coupled Device) image sensor. The imaging element outputs an imaging signal (analog signal) corresponding to the amount of light incident through the color filter. In addition, the imaging element unit 12 converts an analog imaging signal into digital imaging data. And the image pick-up element part 12 outputs the imaging data after conversion according to a synchronizing signal. The synchronization signal includes a vertical synchronization signal indicating one field break and a horizontal synchronization signal indicating one line break, and is supplied from, for example, the image processing unit 20.

  The image processing unit 20 includes a sensor interface (sensor I / F) 21, image processing units 22 and 23, a CPU 24, a memory controller 25, a memory card interface (memory card I / F) 26, and a display interface (display I / F) 27. have. The CPU 24, the sensor I / F 21, and the image processing units 22 and 23 are connected to each other via a CPU data bus 28. The sensor I / F 21, the image processing units 22 and 23, the memory controller 25, the memory card I / F 26, and the display I / F 27 are connected to each other via an image data bus 29.

  The sensor I / F 21 receives imaging data output from the imaging unit 10 and stores it in the memory 32. Therefore, the memory 32 stores imaging data (RAW data) output from the imaging unit 10 and not subjected to image processing. The sensor I / F 21 is an example of a data input unit. The sensor I / F 21 detects the maximum luminance value (maximum value) and the minimum luminance value (minimum value) based on a plurality of pixel data included in the imaging data, and registers the maximum luminance value and the minimum luminance value in the register 21b. To store.

  The image processing unit 22 reads the imaging data stored in the memory 32, performs predetermined image processing on the imaging data, and stores the processed image data in the memory 32. Image processing in the image processing unit 22 includes demosaic processing, contrast correction, γ correction, and gradation correction.

  The image processing unit 23 reads the image data stored in the memory 32, performs predetermined image processing on the image data, and stores the processed image data in the memory 32. The processing in the image processing unit 23 includes, for example, YCbCr data conversion, distortion correction, and encoding processing (JPEG conversion). In FIG. 1, one image processing unit 23 is shown, but the image processing unit 20 may include an image processing unit corresponding to each process.

  The memory card I / F 26 is electrically connected to a memory card 35 that can be attached to and detached from the imaging apparatus. The memory card I / F 26 stores data (for example, compressed image data) stored in the memory 32 in the memory card 35.

  A display device 33 is connected to the display I / F 27. The display device 33 is, for example, a liquid crystal display device (LCD: Liquid Crystal Display). The display device 33 is used for the remaining amount of a battery that is a driving source of the photographing apparatus, a photographing mode, a photographing frame, display of stored image data, and the like. For example, the display I / F 27 reads the image data stored in the memory 32 and outputs the image data to the display device 33.

  The sensor I / F 21, the image processing units 22 and 23, the memory card I / F 26, and the display I / F 27 have memory access controllers (DMAC) 21a to 23a, 26a, and 27a, respectively. The sensor I / F 21 stores image data in the memory 32 via the memory access controller 21a. The image processing units 22 and 23 input and output image data to and from the memory 32 via the memory access controllers 22a and 23a. Similarly, the memory card I / F 26 inputs / outputs image data to / from the memory 32 via the memory access controller 26a. The display I / F 27 outputs the image data read from the memory 32 via the memory access controller 27a to the display device 33.

  The CPU 24 controls the entire image processing unit 20. The CPU 24 performs setting of information necessary for processing to each processing unit, data writing / reading control, and the like. Further, the CPU 24 sets information (parameters) necessary for the operation mode and each process according to the operation of the operation unit 31. The operation unit 31 includes various switches such as a shutter button and a menu button operated by a user, a touch panel, and the like. In addition, the CPU 24 reads the maximum luminance value and the minimum luminance value stored in the register 21 b of the sensor I / F 21 and stores them in the register 22 b of the image processing unit 22.

Next, processing in the sensor I / F 21 will be described.
The sensor I / F 21 obtains the maximum luminance value and the minimum luminance value based on the imaging data.
As shown in FIG. 2A, the imaging data SF of one frame (one screen) includes a plurality of pixel data SG. 2A, pixel data SG arranged in the left-right direction corresponds to a plurality of light receiving units arranged in a first direction (for example, the horizontal direction) of the image sensor, and in FIG. 2A, the vertical direction The pixel data SG arranged in (1) corresponds to a plurality of light receiving units arranged in a second direction (for example, a vertical direction) orthogonal to the first direction of the image sensor. Each pixel data SG includes a value (pixel value) corresponding to the amount of light received by the corresponding light receiving unit. The light receiving unit of the image sensor receives light that has passed through a color filter having a predetermined array (for example, a Bayer array). Accordingly, the pixel value included in each pixel data SG includes color information corresponding to the arrangement and color of the corresponding color filter.

  For example, when each pixel data is distinguished, it is expressed as SG (x, y) according to the arrangement of the light receiving parts. x is the arrangement order in the first direction, and y is the arrangement order in the second direction. For example, in FIG. 2A, the upper left pixel data is represented as SG (1, 1). The pixel data on the right side of the pixel data SG (1,1) is represented as SG (2,1), and the lower pixel data of the pixel data SG (1,1) is represented as SG (1,2). In the imaging unit, the number of the light receiving units arranged in the first direction is m, the number of the light receiving units arranged in the second direction is n, and the lower right pixel data in FIG. n).

  Each pixel data SG includes color information corresponding to the color of the corresponding color filter. For example, a Bayer array color filter includes a red (R) filter, a green (G) filter, and a blue (B) filter. Therefore, the imaging data SF includes a plurality of pixel data SG, and each pixel data SG includes color information of one corresponding color.

  FIG. 2B shows the color information included in each pixel data and the arrangement position of the pixel data. For example, the pixel data SG (1, 1) shown in FIG. 2A includes green (G) color information as shown in FIG. Similarly, the pixel data SG (2, 1) includes red (R) color information. The pixel data SG (1, 2) includes blue (B) color information, and the pixel data SG (2, 2) includes green (G) color information. In the following description, each pixel data may be described using color information. For example, pixel data SG (1, 1) shown in FIG. 2A may be described as pixel data G11 using “G11” shown in FIG.

  A sensor I / F 21 illustrated in FIG. 1 includes a memory (line memory) having a storage capacity corresponding to pixel data of a plurality of lines (for example, two lines). The sensor I / F 21 sequentially stores the pixel data SG in the line memory. And sensor I / F21 calculates the luminance value in each pixel data based on each pixel data SG stored in the line memory. The sensor I / F 21 stores the maximum luminance value (luminance maximum value) and the minimum luminance value (luminance minimum value) in the image data of one frame in the register 21b.

  As described above, in the imaging data output from the imaging element unit 12 including the color filters having a predetermined arrangement (Bayer arrangement), each pixel data includes information of one color corresponding to the arrangement of the color filters. That is, two pieces of color information are lacking in the information of each pixel data. For this reason, the sensor I / F 21 calculates the luminance value in the pixel data SG of interest by referring to surrounding pixel data for insufficient color information in each pixel data SG.

For example, the sensor I / F 21 refers to the color information of the adjacent pixel data R21 and the color information of the pixel data B12 with respect to the pixel data G11 shown in FIG. Calculate based on the formula.
Y11 = 0.29891 × R21 + 0.58661 × G11 + 0.11448 × B12 (1)
The sensor I / F 21 detects the maximum luminance value (maximum luminance value Ymax) and the minimum luminance value (minimum luminance value Ymin) among the luminance values of each pixel data. For example, the sensor I / F 21 stores the maximum luminance value Ymax and the minimum luminance value Ymin that are equal to the luminance value Y11 calculated for the pixel data SG (1, 1) illustrated in FIG. 2A in the register 21b. Next, the sensor I / F 21 compares the luminance value Y21 calculated for the pixel data SG (2, 1) shown in FIG. 2A with the maximum luminance value Ymax and the minimum luminance value Ymin read from the register 21b. For example, when the luminance value Y21 is larger than the maximum luminance value Ymax, the sensor I / F 21 stores the maximum luminance value Ymax equal to the luminance value Y21 in the register 22b.

Next, processing in the image processing unit 22 will be described.
As illustrated in FIG. 3, the image processing unit 22 includes a demosaic unit 41, a contrast correction unit 42, a gamma correction unit (γ correction unit) 43, and a gradation correction unit 44.

  The demosaic unit 41 receives pixel data (RAW data) SR read from the memory 32 shown in FIG. The demosaic unit 41 converts the pixel data SR into pixel data P0 having color information of three colors in each pixel.

  For example, the demosaic unit 41 generates, in each pixel of the pixel data SR, color information that is insufficient with respect to the color information corresponding to the filter based on the surrounding pixel data. Then, the demosaic unit 41 outputs pixel data P0 including color information of one color corresponding to the filter and generated color information of two colors.

  The demosaic unit 41 has a memory (line memory) having a storage capacity corresponding to pixel data of a plurality of lines (for example, 5 lines). The demosaic unit 41 stores the image data read from the memory 32 shown in FIG. 1 in the line memory. Then, the demosaic unit 41 generates insufficient color information in the pixel data by, for example, interpolation using surrounding pixel data. For example, in FIG. 2B, the pixel data G33 has green color information and lacks red color information and blue color information. The demosaic unit 41 corresponds to the position of the pixel data G33 by linear interpolation (interpolation), for example, based on the color information of the pixel data R23 and R43 having red color information among the pixel data adjacent to the pixel data G33. The red color information is generated. Similarly, the demosaic unit 41 generates blue color information corresponding to the position of the pixel data G33 by linear interpolation, for example, based on the color information of the pixel data B32 and B34 having blue color information.

  In addition, the demosaic unit 41 calculates the degree of unevenness RF in each pixel based on the pixel data SR. The demosaic unit 41 generates a kernel including a predetermined number of pixel data based on the pixel data stored in the line memory. The size of the kernel is, for example, 5 × 5 pixels. The demosaic unit 41 pays attention to the center pixel of the kernel, and calculates the unevenness degree RF at the target pixel based on other pixel data included in the kernel. The unevenness RF is a value indicating the color variation in the pixel data included in the kernel. The demosaic unit 41 is an example of a calculation unit.

  The demosaic unit 41 calculates the degree of unevenness RF at the target pixel in the kernel based on predetermined color information among the color information of the plurality of pixel data included in the kernel. For example, the demosaic unit 41 calculates the degree of unevenness RF based on the green (G) color information. Further, the demosaic unit 41 calculates the degree of unevenness RF based on the eight pieces of color information around the pixel of interest. For example, the unevenness degree RF is an average deviation of eight pieces of color information.

  As shown in FIG. 4A, the kernel CU includes 5 × 5 pixel data. In the kernel CU shown in FIG. 4A, the color of the center pixel (target pixel) is green (G). FIG. 4A shows a pixel of interest G22 and pixel data having green color information.

  This kernel CU includes twelve pixel data having green color information in addition to the target pixel G22. The demosaic unit 41 refers to pixel data G20, G11, G31, G02, G42, G13, G33, and G24 surrounding the target pixel G22 among the pixel data included in the kernel CU. Then, the demosaic unit 41 defines the pixel data to be referred to as eight pieces of pixel data G1 to G8.

  For example, the demosaic unit 41 converts the necessary image data into {G1, G2, G3, G4, G5, G6, G7, G8} = {G20, G11, G31, G02 according to the pixel data input order (raster direction). , G42, G13, G33, G24}. Further, the demosaic unit 41 calculates an average value GAVE of the pixel data G1 to G8.

  Then, the demosaic unit 41 calculates the unevenness degree RF according to the following equation (1). The unevenness degree RF is clipped to a value in the range of 0.0 to 1.0.

As shown in the following equation (2), the corrugation degree RF may be calculated by performing a coring process and a scaling process.

In the coring process, pixel data in a predetermined range is flattened (unevenness degree RF = 0.0) with respect to the average value GAVE. COR is a range (threshold value) that is determined to be flat. The scaling process enlarges / reduces the calculation result subjected to the coring process from the average deviation. SCL is a scaling coefficient (gain value). Similar to the above, the unevenness degree RF is clipped to a value in the range of 0.0 to 1.0.

Further, as shown in FIG. 4B, the kernel CU, which is pixel data R22 in which the pixel of interest has red color information, has 12 pieces of pixel data G10, G30, G01, G21, G41 having green color information. , G12, G32, G03, G23, G43, G14, and G34. The demosaic unit 41 defines these 12 pieces of pixel data as 8 pieces of pixel data G1 to G8 as follows. Further, the demosaic unit 41 calculates an average value GAVE of the pixel data G1 to G8.
{G1, G2, G3, G4, G5, G6, G7, G8} = {(G10 + G01) / 2, (G30 + G41) / 2, G21, G12, G32, (G03 + G14) / 2, G23, (G43 + G34) / 2 }
Then, the demosaic unit 41 calculates the degree of unevenness RF according to the above formula (1) or (2) based on the pixel data G1 to G8.

  The result of the operation (multiplication (x2, x4, x8), division (1/2, 1/4, 1/8)) that is a power of 2 is obtained by the shift operation. In the process of calculating, by defining the pixel data as eight pieces of pixel data G1 to G8, it becomes easy to mount an arithmetic circuit.

  The redefinition of the pixel data G1 to G8 includes eight directions centered on the pixel of interest (directions along two orthogonal axes according to the pixel arrangement (the vertical direction and the horizontal direction in FIG. 4A). ) And diagonal information). For example, in FIG. 4A, the pixel data G20 and G24 in the vertical direction, the pixel data G02 and G42 in the horizontal direction, and the pixel data G11, G31, G13, and G33 in the diagonal direction are referred to for the target pixel G22. In FIG. 4B, the vertical pixel data G21 and G23 and the horizontal pixel data G12 and G32 are referred to for the target pixel R22. Further, the pixel data (color information) at the position in the oblique direction calculated based on the peripheral pixel data G10, G01, G20, G41, G03, G14, G43, and G34 is referred to.

  The contrast correction unit 42 outputs pixel data P1a obtained by correcting the contrast of the pixel data P0 output from the demosaic unit 41 based on the maximum luminance value Ymax and the minimum luminance value Ymin stored in the register 22b. In addition, the contrast correction unit 42 generates correction pixel data P1b for correcting gradation loss according to the pixel data P1a. The pixel data P1a is an example of first pixel data, and the correction pixel data P1b is an example of second pixel data.

For example, the contrast correction unit 42 calculates the gain value Cgain and the offset value Coffset based on the maximum luminance value Ymax and the minimum luminance value Ymin using the following equations.
Cgain = DR / (Ymax-Ymin) (3)
Coffset = Ymin (4)
DR is a luminance gradation value (for example, 256).

Then, the contrast correction unit 42 generates pixel data P1a in which each pixel data P0 is subjected to contrast correction based on the gain value Cgain and the offset value Coffset according to the following expression. The pixel data P0 includes color information of each color (R, G, B). The contrast correction unit 42 performs contrast correction for each color and generates pixel data P1a.
P1a = P0 × Cgain−Coffset (5)
Further, the contrast correction unit 42 calculates the correction pixel data P1b by performing contrast correction on a pixel value obtained by adding a predetermined value (for example, “+1”) to the input pixel data P0 according to the following equation.
P1b = (P0 + 1) × Cgain−Coffset (6)
The difference between the pixel data P1a and the correction pixel data P1b indicates a gradation difference that occurs in contrast correction, that is, a gradation skip value.

  The left part of FIG. 7 shows the relationship between the pixel data P0 and the pixel data P1a and P1b in the contrast correction. The values shown in FIG. 7 indicate the case where the gain value Cgain = 1.5 and the offset value Coffset = 0. Since the values of the pixel data P1a and P1b are integers, the decimal part is rounded down.

  For example, when the pixel data P0 is “1”, the contrast correction unit 42 outputs the pixel data P1a of “2”. In addition, the contrast correction unit 42 outputs correction pixel data P1b of “3” corresponding to a value “2” obtained by adding “1” to the value of the pixel data P0. In this case, “2” which is the difference between the pixel data P1a and the correction pixel data P1b is the difference in gradation.

  Further, when the pixel data P0 is “3”, the contrast correction unit 42 outputs the pixel data P1a of “4”. Further, the contrast correction unit 42 outputs correction pixel data P1b of “6” corresponding to a value “4” obtained by adding “1” to the value of the pixel data P0. In this case, “2” which is the difference between the pixel data P1a and the correction pixel data P1b is the difference in gradation.

Next, the γ correction unit 43 will be described.
As illustrated in FIG. 3, the γ correction unit 43 performs gamma correction (γ correction) that corrects color information of each color according to the sensitivity characteristics of the output unit of the image with respect to the pixel data P1a output from the contrast correction unit 42. ) To generate pixel data P2a. Further, the γ correction unit 43 generates γ correction pixel data P2b by γ correcting the correction pixel data P1b output from the contrast correction unit. The pixel data P2a is an example of third pixel data, and the correction pixel data P2b is an example of fourth pixel data.

  The pixel data P1a and the correction pixel data P1b supplied to the γ correction unit 43 have color information of three colors (R, G, B), and the color information of each color is, for example, 8-bit data. Each γ correction unit 43 performs the same process on the color information. Therefore, a circuit and operation corresponding to color information of one color, for example, 8-bit data will be described. For example, one color information included in the pixel data P1a will be described as color data P1a, and one color information included in the correction pixel data P1b will be described as correction color data P1b.

As shown in FIG. 5, the γ correction unit 43 includes a correction table 51, a comparison unit 52, selectors 53 and 54, and a calculation unit 55.
The correction table 51 stores output data for γ correction. The correction table 51 outputs output data stored in an area corresponding to the color data P1a that is input data.

  FIG. 6 shows an example of a γ correction curve. In FIG. 6, the horizontal axis represents the gradation of the input data, and the vertical axis represents the gradation of the output data. The γ value at the image output unit is, for example, “2.2”. The γ correction curve shown in FIG. 6 is a curve having a γ value of, for example, “0.45 (≈1 / 2.2)” in order to generate pixel data having a gradation corresponding to the image output unit.

  The correction table 51 has a plurality of storage areas corresponding to the gradation of the color data P1a (input data). The number of storage areas corresponds to the range (gradation) of the color data P1a. For example, when the color data P1a is 8 bits, the correction table 51 has 256 storage areas. Each storage area stores 8-bit output data. Output data corresponding to the pixel data is stored in the storage area designated by the pixel data. The correction table 51 outputs output data corresponding to the color data P1a (input data). Similarly, the correction table 51 outputs output data corresponding to the correction color data P1b (input data).

  The correction table 51 has a plurality of (four in FIG. 5) memories M1 to M4. The storage capacities of the memories M1 to M4 are set equal to each other. Each of the memories M1 to M4 stores 64 output data according to 8-bit (256 gradations) pixel data.

  For example, the first memory M1 stores output data corresponding to the gradation values 0, 4, 8,..., 56, 60 of the color data P1a. The second memory M2 stores output data corresponding to the gradation values 1, 5, 9,... 57, 61 of the color data P1a. The third memory M3 stores output data corresponding to the gradation values 2, 6, 10,..., 58, 62 of the color data P1a. The fourth memory M4 stores output data corresponding to the gradation values 3, 7, 11,... 59, 63 of the color data P1a.

  The γ correction unit 43 uses the color data P1a as an address signal for the memories M1 to M4, and outputs the color data P2a after γ correction based on the data read from the memories M1 to M4 according to the address signal. Further, the γ correction unit 43 uses the correction color data P1b as an address signal for the memories M1 to M4, and based on the data read from the memories M1 to M4 according to the address signal, the correction color data after γ correction P2b is output.

  For example, the γ correction unit 43 uses the upper 6-bit pixel data P1a [7: 2] of the 8-bit color data P1a [7: 0] as the row address signal RA1 [5: 0], and uses the lower 2 bits. The color data P1a [1: 0] is the column address signal CA1 [1: 0]. The γ correction unit 43 sets the upper 6-bit correction pixel data P1b [7: 2] among the 8-bit correction color data P1b [7: 0] as the row address signal RA2 [5: 0]. The lower 2 bits of color data P1b [1: 0] is used as a column address signal CA2 [1: 0]. The characters in parentheses indicate the range of bits included in the data or signal. For example, [7: 0] indicates that the 7th to 0th bits are included. In FIG. 5, the bit range is omitted. In the following description, the bit range is omitted.

  The number of bits of column address signals CA1 and CA2 corresponds to the number of memories M1 to M4. Column address signals CA1 and CA2 are used to select memories M1 to M4. The number of bits of the row address signals RA1 and RA2 corresponds to the storage capacity of the memories M1 to M4. Row address signals RA1 and RA2 are used for designating storage areas of the memories M1 to M4.

  For example, one storage area included in the memories M1 to M4 is selected by the column address signal CA1 and the row address signal RA1. The data stored in the selected storage area is output data obtained by subjecting the pixel data P1a including the column address signal CA1 and the row address signal RA1 to γ correction. In FIG. 5, the characters shown in the storage areas (indicated by rectangles) of the memories M1 to M4 are the value of the color data P1a (gradation value) and the corresponding output data (numerical values in parentheses). is there.

  The comparison unit 52 compares the column address signal CA1 and the column address signal CA2 with each other, and outputs a detection signal SCT corresponding to the comparison result. The comparison unit 52 outputs a detection signal SCT having a first predetermined value (for example, “1”) when the column address signal CA2 matches the column address signal CA1, and the column address signal CA2 does not match the column address signal CA1. In this case, a detection signal SCT having a second predetermined value (for example, “0”) is output.

  The column address signal CA1 indicates a memory that is accessed according to the color data P1a. The column address signal CA2 indicates a memory that is accessed according to the correction color data P1b. Therefore, the comparison unit 52 detects whether or not the memory corresponding to the color data P1a matches the memory corresponding to the correction color data P1b, and outputs a detection signal SCT corresponding to the detection result.

  The address conversion unit 56 of the correction table 51 generates a row address signal RB2 and a column address signal CB2 for accessing the memories M1 to M4 based on the row address signal RA2, the column address signal CA2, and the detection signal SCT. The detection signal SCT indicates whether or not the row address signal RA1 based on the color data P1a matches the row address signal RA2 based on the color data P1a. The memories M1 to M4 are selected based on the row address signals RA1 and RA2. That is, the detection signal SCT indicates whether the correction data corresponding to the color data P1a and the correction data corresponding to the correction color data P1b are stored in the same memory.

  When the detection signal SCT is “0”, the memory referred to by the color data P1a and the memory referred to by the correction color data P1b are different from each other. In this case, when CT is “0”, the address conversion unit 56 outputs a row address signal RB2 equal to the row address signal RA2 and a column address signal CB2 equal to the column address signal CA2. As a result, the memory corresponding to the correction color data P1b is accessed, and the correction data stored in the memory is output.

  When the detection signal SCT is “1”, the memory referred to by the color data P1a matches the memory referred to by the correction color data P1b. In this case, the address conversion unit 56 accesses the storage area accessed by the correction color data P1b so as to access the storage area adjacent to the front and the storage area adjacent to the rear in the address. An address signal CB2 is generated. The front in the address is the direction in which the address becomes smaller, and the rear in the address is the direction in which the address becomes larger. Further, the storage area adjacent in the address is a storage area having an address different by “1”.

  Therefore, when the detection signal SCT is “1”, the address conversion unit 56 generates a row address signal RB2 and a column address signal CB2 each including two addresses. For the sake of explanation, in order to distinguish these, there are cases where the front address is denoted by “f” and the rear address is denoted by “b”. For example, the row address signal RB2 includes a front address signal RB2f and a rear address signal RB2b.

First, the calculation of the address of the storage area adjacent to the front will be described.
The address conversion unit 56 subtracts “1” from the column address signal CA2 to generate a column address signal CB2f. At this time, when the column address signal CB2f is “−1”, the address conversion unit 56 sets the column address signal CB2f to “3”. The address conversion unit 56 generates a row address signal RB2f equal to the row address signal RA2 when the column address signal CA2 is “1” to “3”. When the column address signal CA2 is “0”, the address conversion unit 56 subtracts “1” from the row address signal RA2 to obtain the row address signal RB2f.

Next, address calculation of the storage area adjacent to the rear will be described.
The address conversion unit 56 adds “1” to the column address signal CA2 to generate the column address signal CB2b. At this time, when the column address signal CB2b is “4”, the address conversion unit 56 sets the column address signal CB2b to “0”. The address conversion unit 56 generates a row address signal RB2b equal to the row address signal RA2 when the column address signal CA2 is “0” to “2”. When the column address signal CA2 is “3”, the address conversion unit 56 adds “1” to the row address signal RA2 to obtain the row address signal RB2b.

  For example, the correction table 51 decodes the 2-bit column address signal CA1 [1: 0] and generates a first selection signal for each of the memories M1 to M4. The correction table 51 decodes the 2-bit column address signal CB2 [1: 0] and generates a second selection signal for each of the memories M1 to M4. Then, the first selection signal and the second selection signal are subjected to a logical operation (logical sum operation) to generate a selection signal for each of the memories M1 to M4. The same applies to the row address signal. As a result, a plurality of memories are selected simultaneously. The selected memory reads correction data in the storage area corresponding to the row address signals RA1 and RB2, and outputs output data D1 to D4.

  The selector 53 is supplied with output data D1 to D4 from the memories M1 to M4. The selector 53 is supplied with a column address signal CA1. The selector 53 selects output data corresponding to the column address signal CA1 from the output data D1 to D4, and outputs color data P1a equal to the selected output data. For example, when the column address signal CA1 is “00”, the selector 53 selects the output data D1 of the memory M1 and outputs color data P1a equal to the output data D1. The output data D1 to D4 in the selector 53 is an example of first correction data.

  The selector 54 is supplied with output data D1 to D4 from the memories M1 to M4. The selector 54 is supplied with a column address signal CA2 and a detection signal SCT. The selector 54 outputs two output data SD1 and SD2 corresponding to the output data D1 to D4 of the memories M1 to M4 based on the column address signal CA2 and the detection signal SCT.

  When the detection signal SCT is “0”, the selector 54 selects one output data corresponding to the column address signal CA2 from the output data D1 to D4, and selects two output data SD1 and SD2 equal to the selected data. Output. For example, when the column address signal CA2 is “01”, the selector 54 selects the output data D2 of the memory M2 and outputs output data SD1 and SD2 equal to the output data D2. Output data D1 to D4 in the selector 54 is an example of second correction data.

  When the detection signal SCT is “1”, the selector 54 selects two output data among the output data D1 to D4 based on the column address signal CA2, and the output data SD1 that is equal to the two selected output data, respectively. , SD2 is output. At this time, the two output data selected by the selector 54 are the output data corresponding to the front address and the output data corresponding to the rear address described in the correction table 51 described above.

  The selector 54 obtains a value obtained by subtracting “1” from the column address signal CA2. At this time, when the result value is “−1”, the selector 54 sets the result value to “3”. Then, the selector 54 selects memory output data corresponding to the result value. For example, when the column address signal CA2 is “1”, the calculation result is “0”. Based on the result, the selector 54 selects the output data D1 of the memory M1 according to the column address signal CA2 of “0”. Similarly, when the calculation results are “1” to “3”, the output data D2 to D4 of the memories M2 to M4 are selected. Then, the selector 54 outputs output data SD1 that is equal to the selected data.

  The selector 54 obtains a value obtained by adding “1” to the column address signal CA2. At this time, when the result value is “4”, the selector 54 sets the result value to “0”. Then, the selector 54 selects memory output data corresponding to the result value. Then, the selector 54 outputs output data SD2 that is equal to the selected data.

The calculation unit 55 calculates the output data SD1 and SD2 output from the selector 54 and generates correction color data P2b. The calculation in the calculation unit 55 is, for example, linear interpolation calculation (average calculation). This linear interpolation calculation is expressed by the following equation.
P2b = (SD1 + SD2) / 2 (7)
For example, when the detection signal SCT is “0”, the selector 54 outputs two output data SD1 and SD2 equal to one output data selected according to the column address signal CA2. Therefore, the correction color data P2b calculated by the above equation (7) is equal to one output data selected by the selector 54. That is, output data equal to the correction data corresponding to the column address signal CA2 is output from the arithmetic unit 55.

  When the detection signal SCT is “1”, two correction data adjacent to the correction color data P <b> 1 b are output from the correction table 51. The calculation unit 55 linearly interpolates these two correction data (second correction data) to generate correction color data P2b. The correction color data P2b is a value that is equal to a value obtained by γ-correcting the correction color data P1b or a value close to a value obtained by γ-correcting the correction color data P1b.

  For example, as shown in FIG. 7, when the color data P1a is “1”, the correction table 51 outputs output data “20” according to the color data P1a. As shown in FIG. 5, the color data P1a indicates the memory M2. When the correction color data P1b is “3”, the correction color data P1b indicates the memory M4. Therefore, the detection signal SCT is “0”, and the correction table 51 outputs “33” output data as shown in FIG.

  As another example, when the color data P1a is “1” and the correction color data P1b is “5”, the color data P1a and the correction color data P1b both indicate the memory M2, as shown in FIG. Therefore, the detection signal SCT is “1”. At this time, the correction table 51 outputs the output data D2 of “20” according to the color data P1a. The correction table 51 outputs “38” output data D1 before the correction color data P1b and “38” output data D3 after the correction color data P1b. The selector 54 selects the output data D1 and D3 and outputs the output data SD1 and SD2. Then, the computing unit 55 outputs the correction color data P2b of “42” based on the output data SD1 and SD2. At this time, the value obtained by performing γ correction on the correction color data P1b is “42”. Therefore, the γ correction unit 43 generates correction color data P2b that is equal to the value obtained by γ correction.

  As another example of the calculation, a case where the correction color data P1b is “6” will be described. At this time, the correction table 51 outputs “42” output data D2 before the correction color data P1b and “49” output data D4 after the correction color data P1b. The selector 54 selects the output data D2 and D4 and outputs the output data SD1 and SD2. Then, the computing unit 55 outputs “45” correction color data P2b based on the output data SD1 and SD2. The fractional part is rounded down. At this time, the value obtained by performing γ correction on the correction color data P1b is “46”. Accordingly, the γ correction unit 43 generates correction color data P1b that is close to the value obtained by γ correction.

Next, the gradation correction unit 44 will be described.
As shown in FIG. 3, the gradation correction unit 44 is a pixel obtained by performing gradation correction on the pixel data P2a with the correction pixel data P2b based on the pixel data P2a and the correction pixel data P2b output from the γ correction unit 43. Data P3 is generated. The pixel data P3 is an example of fifth pixel data. Further, the gradation correction unit 44 generates pixel data P3 obtained by performing gradation correction on the pixel data P2a with the correction pixel data P2b based on the unevenness degree RF generated by the demosaic unit 41.

As illustrated in FIG. 8, the gradation correction unit 44 includes a pseudo random number generation unit 61 and a correction calculation unit 62.
The pseudo random number generation unit 61 is supplied with the set value Ss. The set value Ss is stored, for example, in the register 22b shown in FIG. The pseudo random number generation unit 61 generates a random number RP using the set value Ss as a reference value. The pseudo-random number generator 61 generates a random number RP by, for example, a mixed congruential method. The random number RP becomes a value within a predetermined range with a substantially equal probability. This range is set in accordance with, for example, the maximum value of gradation skip caused by contrast correction and γ correction.

The correction calculation unit 62 generates pixel data P3 based on the pixel data P2a and P2b, the random number RP, and the unevenness level RF. For example, the correction calculation unit 62 calculates the difference value ΔP between the pixel data P2a and P2b, and adds the value resulting from the remainder calculation of the random number RP using the difference value ΔP to the pixel data P2a to generate the pixel data P3. That means
ΔP = P2b−P2a
P3 = P2a + (RP% ΔP) (8)
Where% is the remainder operation,
Thus, the pixel data P3 is calculated. Therefore, the remainder calculation result by the random number RP and the difference value ΔP is a correction amount for the pixel data P2a.

Further, the correction calculation unit 62 may generate the pixel data P3 based on the following formula based on the unevenness degree RF.
P3 = P2a + (R% ΔP) × (1.0−RF) (9)
The unevenness level RF is added to the value (= R% ΔP) added to the pixel data P2a. For example, when the unevenness degree RF is “0”, the remainder calculation result of the random number RP and the difference value ΔP is added to the pixel data P2a to generate the pixel data P3. Further, when the unevenness level RF is “1”, pixel data P3 equal to the pixel data P2a is generated. That is, the unevenness level RF sets the maximum value of the correction amount added to the pixel data P2a.

  For example, in the case of image data of a portion obtained by photographing the sky, the unevenness level RF is “0” or a value close to “0”. As described above, the gradation skip is conspicuous in the flat portion where the difference between the target pixel and the surrounding pixels is small. On the other hand, in a portion where the difference between the pixel of interest and the surrounding pixels is large, the gradation skip is hardly noticeable. Therefore, by setting the maximum value of the correction amount to be added according to the unevenness level RF, an image with less noticeable gradation can be obtained.

Next, the operation of the above imaging device will be described.
FIG. 9A shows a histogram of luminance values in the image data that did not perform photometry well. The vertical axis in FIG. 9A represents the number of pixels (number of pixels) having each luminance value.

  In the imaging data having such a histogram, a shadow portion having a small luminance value and a highlight portion having a large luminance value are not used, and a so-called sleepy image without sharpness is obtained. Thus, an image in which the dynamic range is not effectively used has a low contrast and cannot be said to be a good-looking image. The same applies to a dark image in which there are many shadow portions and no highlight portion is used, and a bright image in which there are many highlight portions and no shadow portion is used.

  FIG. 9B is a histogram of luminance values in pixel data obtained by contrast correction. The vertical axis in FIG. 9B is the number of pixels having each luminance value (number of pixels). In this manner, a comb-like histogram in which pixels having luminance values periodically do not exist is obtained according to the gain value Cgain in contrast correction. In an image based on such imaged data, a striped pattern generated in a portion with a small change in luminance such as the sky or the skin of a person is conspicuous.

  In the γ correction, as in the γ correction curve shown in FIG. 6, the input data (P1a) whose gradation value is “0” to “50” and the output data whose gradation value is “0” to “122” are used. Correction to (P2a). Therefore, in the shadow portion where the gradation value is close to “0”, the gradation difference of the output data (P2a) is larger than the gradation difference of the input data (P1a).

  As shown in FIG. 3, the gradation correction unit 44 adds a correction amount randomly generated according to the difference value ΔP between the pixel data P2a output from the γ correction unit 43 and the correction pixel data P2b to the pixel data P2a. (Addition) is performed to generate pixel data P3. As a result, as shown in FIG. 9C, the pixel data P3 after gradation correction is a histogram in which gradation skip is inconspicuous. The calculation process in the gradation correction unit 44 is the same calculation process as the addition of noise. For this reason, it feels tentatively rough with respect to the image before correction, but the striped pattern becomes less worrisome compared to an image where gradation correction is not performed on a flat portion of the image. In this way, deterioration of image quality can be suppressed.

As described above, according to the present embodiment, the following effects can be obtained.
(1) The contrast correction unit 42 includes pixel data P1a obtained by correcting the contrast of the pixel data P0 based on the maximum luminance value Ymax and the minimum luminance value Ymin, and pixel data obtained by adding a predetermined value “1” to the pixel data P0. Correction pixel data P1b with corrected contrast is generated. The γ correction unit 43 performs pixel data P2a on which pixel data P1a is subjected to gamma correction (γ correction) for correcting color information of each color in accordance with sensitivity characteristics of an image output unit, and correction pixel data P1b. Correction pixel data P2b is generated by performing γ correction. The gradation correction unit 44 generates pixel data P3 obtained by performing gradation correction on the pixel data P2a according to the difference value ΔP between the pixel data P2a and the correction pixel data P2b. The difference value ΔP corresponds to a gradation skip caused by contrast correction and γ correction. By performing tone correction on the pixel data P2a after γ correction according to the difference value ΔP, it is possible to suppress deterioration in image quality.

  (2) The gradation correction unit 44 generates pixel data P3 obtained by performing gradation correction on the pixel data P2a with the correction pixel data P2b based on the unevenness level RF. For example, in the case of image data of a portion obtained by photographing the sky, the unevenness level RF is “0” or a value close to “0”. As described above, the gradation skip is conspicuous in the flat portion where the difference between the target pixel and the surrounding pixels is small. On the other hand, in a portion where the difference between the pixel of interest and the surrounding pixels is large, the gradation skip is hardly noticeable. Therefore, by setting the maximum value of the correction amount to be added according to the unevenness level RF, it is possible to obtain an image in which gradation skip is not noticeable.

In addition, you may implement each said embodiment in the following aspects.
-You may make it produce | generate image data (RAW data) using the imaging optical system containing the imaging sensor in which three light-receiving parts corresponding to each color were formed along the orthogonal | vertical direction with respect to the incident plane of light. Image data corresponding to one pixel includes color information of three colors (R, G, B). For this reason, the sensor I / F 21 shown in FIG. 1 calculates the luminance value in the pixel data based on the color information of the three colors included in one pixel data. In this case, the demosaic process in the demosaic unit 41 shown in FIG. 3 can be omitted.

In the demosaic unit 41 shown in FIG. 3, the unevenness degree RF may be calculated based on the generated pixel data SR including RGB color information.
Further, in the demosaic unit 41 illustrated in FIG. 3, the unevenness degree RF may be calculated using a calculation that can show a variation from the average, such as a dispersion calculation or a standard deviation.

  -For the above embodiment, the kernel size may be changed as appropriate, for example, 3 × 3, 7 × 7, or the like. Alternatively, the unevenness degree RF may be calculated using kernels having different vertical and horizontal sizes (for example, 5 × 7, 7 × 5).

  In the γ correction unit 43 shown in FIG. 5, the comparison unit 52 is omitted, and the correction data read from the two memories is calculated by the calculation unit 55 based on the correction color data P1b to generate the correction color data P2b. You may make it do.

-You may change suitably the calculation in the calculating part 55 shown in FIG. For example, the correction color data P2b may be generated using the α blend operation as in the following equation.
P2b = SD1 × α + SD2 × (1−α)
α is a coefficient, and this coefficient α is obtained by a lookup table (LUT: Look Up Table) that stores data corresponding to the γ correction curve shown in FIG. 6, for example. The data of the reference table can be calculated and stored by a logic circuit, for example, by approximating a γ correction curve to a polygonal line. By using the coefficient α corresponding to the γ correction curve, the correction color data P2b can be obtained with high accuracy.

  In the contrast correction unit 42, the value obtained by adding “1” to the pixel value of the pixel data P1a is used as the pixel value of the correction pixel data P1b, but a value obtained by subtracting “1” may be used. Even in this way, it is possible to obtain the difference value ΔP corresponding to the gradation skip occurring in the contrast correction and the γ correction, and the gradation correction can be performed.

  In the gradation correction unit 44, the maximum correction amount is determined according to the degree of unevenness RF. The necessity of correction may be determined based on the unevenness level RF. For example, the unevenness degree RF may be compared with a predetermined threshold value, and correction may be performed when the unevenness degree RF is greater than the threshold value. The threshold value is set according to the unevenness level RF and the gradation value.

  For example, when the unevenness degree RF is “0” to “1” as in the above embodiment, the threshold value is set to “0.5”, for example. When the unevenness degree RF is smaller than the threshold value, the pixel data P3 is generated by the above formula (8) or formula (9). On the other hand, when the unevenness degree RF is equal to or greater than the threshold value, pixel data P3 having a value equal to the pixel data P2a is generated. Even in this way, it is possible to correct the gradation skip in the flat portion and suppress the deterioration of the image quality.

  -You may change suitably the bit number of the color information of each color contained in pixel data. For example, 10 bits, 12 bits, etc. The number of bits of color information corresponds to the number of gradations. Therefore, the storage capacity of the correction table 51 shown in FIG. 5 and the number of bits of the row address signals RA1, RA2, RB2 are changed according to the number of bits of color information.

  In the above embodiment, the imaging unit 10 including a Bayer array color filter is used, but an imaging element unit including another array color filter may be used. Further, the pixel arrangement direction is not limited to two orthogonal axes. Furthermore, an image sensor in which pixels of each color are formed in the depth direction of the chip may be used. Further, an imaging element unit including filters of colors other than red (R), blue (B), and green (G) may be used. The filter is not limited to the primary color filter, and a complementary color filter may be used.

21 Sensor interface (Sensor I / F)
21b register 22 image processing unit 22b register 41 demosaic unit 42 contrast correction unit 43 gamma correction unit (γ correction unit)
44 gradation correction unit 51 correction table 52 comparison unit 53, 54 selector 55 calculation unit 56 address conversion unit 61 pseudo random number generation unit 62 correction calculation unit M1-M4 memory D1-D4 output data SCT detection signal P1a, P2a color data (pixel) data)
P1b, P2b Correction color data (correction pixel data)
P3 Output pixel data Cgain Gain value Coffset Offset value RF Unevenness RP Random number Ymax Maximum luminance value Ymin Minimum luminance value

Claims (10)

First pixel data obtained by contrast-correcting the pixel data based on a gain value and an offset value corresponding to a maximum luminance value and a minimum luminance value of imaging data of one screen including a plurality of pixel data, and a predetermined value for the pixel data A contrast correction unit that generates second pixel data in which the value obtained by adding the values is subjected to contrast correction;
A gamma correction unit that generates third pixel data obtained by gamma-correcting the first pixel data according to a correction table; and fourth pixel data obtained by gamma-correcting the second pixel data according to the correction table;
A gradation correction unit that generates fifth pixel data obtained by correcting the third pixel data according to a difference value (ΔP) between the third pixel data and the fourth pixel data;
An image processing apparatus. The gradation correction unit
A random number generator for generating random numbers;
A correction calculation unit that adds the correction amount calculated based on the random number and the difference value to the third pixel data to generate the fifth pixel data;
The image processing apparatus according to claim 1. A calculation unit that calculates the degree of unevenness in the pixel of interest based on pixel data around the pixel of interest;
The correction calculation unit generates the fifth pixel data by adding a value based on the correction amount to the third pixel data according to the degree of unevenness;
The image processing apparatus according to claim 2. The gamma correction unit is
Including at least three memories storing different correction data;
Generating the third pixel data according to the first correction data read from one of the memories according to the first pixel data;
Calculating the second correction data read from each of the two memories different from the memory from which the first correction data is read according to the second pixel data to generate the fourth pixel data;
The image processing apparatus according to any one of claims 1 to 3. The two memories are memories corresponding to pixel data before and after the second pixel data;
The image processing apparatus according to claim 4. The gamma correction unit is
It is determined whether or not the memory corresponding to the first pixel data and the memory corresponding to the second pixel data match, and if they match, the second correction read from each of the two memories Calculate the data to generate the fourth pixel data, and if they do not match, generate the fourth pixel data according to the second correction data read from the memory according to the second pixel data To do,
The image processing apparatus according to claim 4, wherein: The image data output from the imaging unit is stored in a memory, and has a data input unit that detects a luminance maximum value and a luminance minimum value in the image data,
The contrast correction unit generates the first pixel data and the second pixel data based on image data read from the memory;
The image processing apparatus according to claim 1, wherein: Pixel data including color information corresponding to the color filter included in the imaging unit among a plurality of colors is generated based on the color information included in the surrounding pixels. Including a demosaic unit that generates pixel data including
The image processing apparatus according to claim 1, wherein:   The image processing apparatus according to claim 8, wherein the demosaic unit generates a degree of unevenness based on the pixel data including color information of the one color. An image processing method executed by an image processing apparatus that processes a plurality of pixel data read from a memory,
A first pixel data obtained by contrast-correcting the pixel data based on a gain value and an offset value corresponding to a maximum luminance value and a minimum luminance value of the plurality of pixel data, and a value obtained by adding a predetermined value to the pixel data. Generating second pixel data with contrast correction;
Generating third pixel data in which the first pixel data is gamma-corrected according to the correction table, and fourth pixel data in which the second pixel data is gamma-corrected according to the correction table;
Generating fifth pixel data obtained by correcting the third pixel data according to a difference value (ΔP) between the third pixel data and the fourth pixel data;
An image processing method characterized by the above.