JP2009200762A - Image processing circuit, semiconductor device, and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing circuit capable of improving the image quality and visibility for an optional region included in an input image. <P>SOLUTION: The image processing circuit comprises: a correction effective region setting means for setting the correction effective region of the input image; a luminance conversion coefficient calculation means for dividing only the correction effective region of the input image into a plurality of areas and then calculating a luminance conversion coefficient for each area; and a luminance conversion processing section for performing luminance conversion processing corresponding to the luminance conversion coefficient for each area on pixels configuring each area. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing circuit that generates a desired output image by performing luminance conversion processing (luminance dynamic range correction processing) on an input image, a semiconductor device formed by integrating the image processing circuit, and an image processing device using the image processing circuit. Is.

  Conventionally, luminance conversion processing is generally used as one of image processing for making an input image look more beautiful. In this luminance conversion process, for example, a luminance conversion coefficient common to the entire image region is calculated based on a luminance histogram in the entire image, and the luminance conversion is performed by multiplying each pixel by the luminance conversion coefficient.

  More specifically, first, a luminance histogram (frequency of each luminance range, that is, distribution of the number of pixels belonging to each predetermined luminance range) for the entire input image is obtained. Based on this luminance histogram, for each luminance range in the input image, a higher luminance frequency is assigned to the output image, while a lower luminance frequency is assigned to the output image. As described above, the luminance conversion coefficient is calculated. Therefore, the luminance conversion coefficient is a separate value for each luminance or for each predetermined luminance range.

  As a result, the contrast (brightness difference) becomes more ambiguous in the part of the luminance range that is less frequent in the input image, but the contrast is clearer in the part of the luminance range that is frequent in the input image. It becomes. Therefore, when viewed as an entire image, the contrast can be made clearer than at the time of input.

In addition, patent documents 1-4 etc. can be mentioned as a prior art relevant to the above.
JP-T-2004-530368 JP 2000-013625 A JP 2000-036043 A JP 2004-145399 A

  Certainly, with the above-described conventional image processing apparatus, when viewed as an entire image, the contrast of the output image can be made clearer than the input image, so that the image quality and visibility can be improved. .

  However, in the conventional image processing apparatus, since the entire input image is fixed as the target of the luminance conversion process, the luminance conversion process may be performed even on an area not intended by the user, and the image quality and visibility are sufficiently high. There was a risk that it could not be improved.

  In view of the above problems, the present invention provides an image processing circuit capable of improving the image quality and visibility of an arbitrary area included in an input image, a semiconductor device in which the image processing circuit is integrated, and the use thereof. An object of the present invention is to provide an image processing apparatus.

  In order to achieve the above object, an image processing circuit according to the present invention includes a correction effective area setting unit that sets a correction effective area of an input image; of the input image, only the correction effective area is divided into a plurality of areas. Then, a luminance conversion coefficient calculation unit that calculates a luminance conversion coefficient for each area; a luminance conversion processing unit that performs a luminance conversion process according to the luminance conversion coefficient for each area on the pixels that constitute each area; And (a first configuration).

  In the image processing circuit having the first configuration, the correction effective area setting unit has a configuration (second configuration) including a register for storing a setting value for determining the correction effective area. Good.

  In the image processing circuit having the second configuration, the set value includes a register value indicating the start position of the correction effective area and a register value indicating the pixel size of the correction effective area. A configuration (third configuration) is preferable.

  The semiconductor device according to the present invention has a configuration (fourth configuration) in which the image processing circuits having any one of the first to third configurations are integrated.

  An image processing apparatus according to the present invention includes a semiconductor device having the fourth configuration described above and an image input source that supplies the input image to the semiconductor device (fifth configuration). Has been.

  In the image processing circuit according to the present invention, a semiconductor device formed by integrating the image processing circuit, and an image processing device using the image processing circuit, an arbitrary correction image can be included in the input image by arbitrarily setting the correction effective area of the input image. It is possible to improve the image quality and visibility of these areas.

  In the following, a detailed description will be given by taking as an example a case where the present invention is applied to an image processing apparatus that performs luminance conversion processing on an image of each frame obtained by moving image capturing.

  First, a first embodiment of an image processing apparatus according to the present invention will be described in detail.

  FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus according to the present invention.

  As shown in this figure, the image processing apparatus 1 of the present embodiment includes an imaging unit 10, a luminance conversion processing unit 11, a luminance conversion coefficient calculation unit 12, a luminance conversion coefficient storage unit 13, and an output unit 14. It consists of

  The imaging unit 10 includes a predetermined lens group, an imaging device (for example, a CMOS [Complementary Metal-Oxide Semiconductor] sensor or a CCD [Charge Coupled Devices] sensor), and the like, and forms an optical image of the subject to form the subject. Imaging processing (for example, moving image capturing at 30 frames per second). Note that image data relating to each frame is sequentially output to the luminance conversion processing unit 11 and the luminance conversion coefficient calculation unit 12.

  The luminance conversion processing unit 11 performs luminance conversion processing on the image data (input image) for each frame input from the imaging unit 10 and outputs the image data. This luminance conversion process is performed by converting the luminance of each pixel in the input image according to the luminance conversion coefficient stored in the luminance conversion coefficient storage unit 13.

  The luminance conversion coefficient calculation unit 12 calculates the luminance conversion coefficient used for the luminance conversion processing based on the image data (input image) for each frame input from the imaging unit 10. The content of the luminance conversion coefficient and the method for calculating the luminance conversion coefficient will be described later.

  The luminance conversion coefficient storage unit 13 stores the luminance conversion coefficient calculated by the luminance conversion coefficient calculation unit 12 at least until the luminance conversion process related to the next frame is executed. This stored content is used in the luminance conversion processing in the luminance conversion processing unit 11.

  The output unit 14 includes a display such as an LCD [Liquid Crystal Display] (for example, an in-vehicle monitor), and sequentially displays output images that have been subjected to luminance conversion processing by the luminance conversion processing unit 11.

  The image processing apparatus 1 having the above configuration performs a luminance conversion process based on a luminance conversion coefficient on an input image obtained by moving image capturing, and displays this on a display.

  Of the above-described components, the luminance conversion processing unit 11, the luminance conversion coefficient calculation unit 12, and the luminance conversion coefficient storage unit 13 may be integrated in a semiconductor device.

  Next, the overall flow of image processing in the image processing apparatus 1 will be described with reference to FIG.

  FIG. 2 is a diagram for explaining the overall flow of image processing in the image processing apparatus 1.

  As shown in this figure, when an input image related to the nth frame arrives, the luminance conversion coefficient calculation unit 12 calculates a luminance conversion coefficient based on the input image. As a result, a luminance conversion coefficient determined based on the nth frame is obtained, and is temporarily stored in the luminance conversion coefficient storage unit 13.

  On the other hand, the luminance conversion processing unit 11 is determined based on the (n−1) th frame already stored in the luminance conversion coefficient storage unit 13 for the input image related to the nth frame that has arrived. Luminance conversion processing is performed using the luminance conversion coefficient.

  More specifically, for a pixel at coordinates (i, j) in the nth frame, the luminance before luminance conversion processing is Iij (n), the luminance after luminance conversion processing is Oij (n), and (n−1) ) If the luminance conversion coefficient determined for the pixel at coordinates (i, j) based on the first frame is Tij (n−1), the luminance related to each pixel in each frame is (1) Conversion processing is performed based on the expression. Then, the output image that has been subjected to the luminance conversion processing by the luminance conversion processing unit 11 is output through the output unit 14.

Oij (n) = Tij (n−1) × Iij (n) (1)
As described above, the image processing apparatus 1 according to the present embodiment uses the luminance conversion coefficient determined based on the (n−1) th frame for the input image related to the nth frame. It is set as the structure which processes. Therefore, it is possible to execute the nth luminance conversion process and image output without waiting for the calculation of the nth luminance conversion coefficient. As a result, it is possible to output an input image that has been subjected to luminance conversion processing as early as possible, and output an image close to real time.

  Note that although it is possible to perform the luminance conversion process on the input image related to the nth frame using the luminance conversion coefficient determined based on the (n−2) th and previous frames, It should be noted that the accuracy of the conversion process becomes a problem when the luminance conversion coefficient based on the old frame is used. This problem is particularly noticeable when the moving image has a large movement. For example, when a still image is handled as an input image, the luminance conversion process is performed on the input image related to the nth frame using a luminance conversion coefficient determined based on the nth frame. You may make it do.

  Next, the content of the luminance conversion coefficient calculation process will be described with reference to the flowchart of FIG.

  FIG. 3 is a flowchart for explaining the luminance conversion coefficient calculation processing in the image processing apparatus 1.

  First, area division is performed on an input image for one frame (step S11).

  FIG. 4 is a diagram for explaining a mode of area division in the image processing apparatus 1.

  In the example of FIG. 4, it is assumed that one frame is composed of 40 × 64 pixels (= 2560), and one area is 8 × 8 pixels in size. Accordingly, the total number of areas is 5 vertical areas × 8 horizontal areas = 40 areas. Each area will be referred to as A0, A1,..., An,.

  When the area division is completed, the area-specific conversion coefficient is calculated (step S12 in FIG. 3). This area-specific conversion coefficient is determined for each area, and is common to each pixel in the area.

  Note that the method for calculating the conversion coefficient for each area is the same as the method for calculating the conversion coefficient for the entire frame, and a detailed description thereof is omitted here. This is different from the conventional configuration in that after a luminance histogram (appearance frequency of pixels belonging to each luminance or each luminance range) is calculated, a conversion coefficient is calculated based on the luminance histogram for each area. According to the area-specific conversion coefficient calculated in this way, for each divided area, a wider luminance range in the output image is assigned to a higher luminance range in the input image.

  In step S12 of FIG. 3, area-specific conversion coefficients are calculated for all areas in the image area. That is, a unique area-specific conversion coefficient is determined for each area. Then, the area-specific conversion coefficient may be adopted as the luminance conversion coefficient as it is, and the luminance conversion process of the input image may be executed. As a result, even when a portion having a large luminance difference as compared with the entire image partially exists, it is possible to obtain a good contrast including the portion.

  However, the area-specific conversion coefficient does not basically consider luminance information outside the area, that is, other areas. For this reason, the difference in luminance is conspicuous (brightness distribution has a high frequency) at the boundary between areas, and an image that is not good in terms of smoothness may be output. Therefore, in this embodiment, filter processing or the like is performed on the area-specific conversion coefficients calculated for each area. The contents of this process will be specifically described.

  First, prior to execution of filter processing or the like, a virtual area having the same scale as the above-described area is set outside the image area.

  FIG. 5 is a diagram for explaining a state in which virtual areas (A40 to A69 in the figure) are set.

  Then, virtual area-specific conversion coefficients (hereinafter referred to as virtual coefficients) are set for these virtual areas A40 to A69 (step S13 in FIG. 3). As will be described later, this virtual coefficient is used to allow normal filtering processing to be performed on an area near the outer edge of the image area, and takes the same form as the area-specific conversion coefficient. Is.

  The virtual coefficient is set with reference to each area adjacent to the virtual area (each area located at the outer edge of the image area) with reference to the area-specific conversion coefficient of the area existing at a symmetric position with the virtual area. . For example, since A41 is symmetric with A8 with respect to A0, the virtual coefficient of A41 has the same value as the area-specific conversion coefficient of A8. Similarly, since A42 is symmetric with A9 with respect to A1, the virtual coefficient of A42 is set to the same value as the area-specific conversion coefficient of A9. On the other hand, since A40 located at the four corners is symmetrical with A9 with respect to A0, the virtual coefficient of A40 has the same value as the area-specific conversion coefficient of A9.

  In addition to the above, the setting method of the virtual coefficient may be set to be symmetric with respect to the outer edge of the image area (thick line in FIG. 5), for example. In this case, since A41 is symmetrical with A0 with respect to the outer edge of the image area, the virtual coefficient of A40 is set to the same value as the area-specific conversion coefficient of A0. Similarly, since A42 is symmetric with A1 with respect to the outer edge of the image area, the virtual coefficient of A42 is set to the same value as the area-specific conversion coefficient of A1. On the other hand, A40 located at the four corners is symmetric with A0 with respect to the outer edge of the image area, so the virtual coefficient of A40 is the same value as the area-specific conversion coefficient of A0.

  After setting the virtual coefficient for each virtual area in this way, next, a process (filter process) for applying a low-pass filter in the image space is performed on the area-specific conversion coefficient in each area in the image area (FIG. 5). 3 step S14).

  FIG. 6 is a diagram for explaining an aspect of the low-pass filter.

  According to this, when a filter is applied to a certain area of interest, the area-specific conversion coefficients of areas existing in a predetermined range around the area of interest (in this case, the four areas on the top, bottom, left, and right) are used. However, when a part of the predetermined range protrudes from the image area, the virtual coefficient assigned to the virtual area at that position is assumed to be the area-specific conversion coefficient.

  For example, the area-specific conversion coefficient a ′ (0) after the filter processing in the area A0 is as follows. Note that a (n) represents the area-specific conversion coefficient in the area An (n = 0, 1, 8, 41, 50) before the filter processing. Further, a (41) and a (50) are virtual coefficients in the virtual areas A41 and A50, respectively.

a ′ (0) = a (0) / 2 + [a (1) + a (8) + a (41) + a (50)] / 8
Such a filtering process can reduce the difference in luminance even when the difference in luminance is conspicuous at the boundary between areas. In the present embodiment, the low-pass filter shown in FIG. 6 has been described. However, the area-specific conversion coefficient related to the area up to which range is considered, and the weighting for each area in the filter is performed. Various aspects can be adopted.

  The area-specific conversion coefficient that has been subjected to the above-described filter processing may be directly used as the luminance conversion coefficient, and the luminance conversion process of the input image may be executed. However, in this embodiment, in order to further obtain the smoothness of the output image, a pixel-specific conversion coefficient determined for each pixel (not necessarily common to each pixel even in the same area) is calculated (step of FIG. 3). S15), and this is applied as a luminance conversion coefficient (step S16 in FIG. 3).

  Here, a calculation method (bilinear calculation) of the conversion coefficient for each pixel will be described with reference to FIG.

  FIG. 7 is a diagram for explaining bilinear calculation.

  In determining the pixel-specific conversion coefficient for a certain target pixel, attention is paid to four areas related to the vicinity of the pixel. For example, when the pixel P in FIG. 7 is the target pixel, attention is paid to the areas A0, A1, A8, and A9. In other words, four areas are selected so that each area shares one vertex and the pixel of interest is located inside a quadrilateral formed by connecting the centers of the areas.

  The pixel-specific conversion coefficients of the pixel of interest P are the area-specific conversion coefficients [a ′ (0), a ′ (1), a ′ (8), a ′ ( 9)], the position of the target pixel P, and the center position in each of the four areas.

  More specifically, as shown in FIG. 7, the horizontal distance between the areas is X, the vertical distance between the areas is Y, the horizontal distance between the center of the area A0 and the target pixel P is a, and the vertical distance is the same. When the distance in the direction is b, the pixel-by-pixel conversion coefficient p (P) of the target pixel P is obtained as follows.

p (P) = {a ′ (0) × (X−a) × (Y−b)
+ A ′ (1) × a × (Y−b)
+ A ′ (8) × (X−a) × b
+ A ′ (9) × a × b] / (X × Y)
Even in the bilinear calculation, if the target pixel is in the vicinity of the outer edge of the image area, it is conceivable that a part of the four areas of interest protrudes from the image area. Even in such a case, a normal bilinear calculation can be performed by setting a virtual area and a virtual coefficient as in the case of the filter processing described above.

  With this method, pixel-specific conversion coefficients are calculated for all pixels in the image area, and these are used as luminance conversion coefficients for the corresponding pixels. Accordingly, the pixel-specific conversion coefficient is calculated based on the area-specific conversion coefficient so that the luminance change is smooth including the boundary portion between the areas. As a result, it is possible to avoid the occurrence of a discontinuous portion of luminance change as much as possible, and it becomes easy to output a more beautiful image.

  When bilinear calculation is used as described above, the pixel-specific conversion coefficient is usually calculated so that the difference in luminance at the boundary between areas becomes smaller. However, another calculation means may be applied instead of the bilinear calculation or in addition to the bilinear calculation in order to surely reduce the luminance difference at the boundary between the areas.

  In addition, the above-described luminance conversion coefficient storage unit 13 stores only the area-specific conversion coefficients (coefficients for each area), and finally the pixel-specific conversion coefficients (coefficients for each pixel) to be adopted as the luminance conversion coefficients. May be calculated when the luminance conversion process is executed. In this way, the memory capacity of the luminance conversion coefficient storage unit 13 can be reduced as compared with that storing pixel-specific conversion coefficients.

  As for the above-mentioned “pixel”, literally a single pixel is regarded as a “pixel”, and for example, a pixel group composed of RGB (red, green, and blue) pixels is considered as a unit and treated as a “pixel”. It may be. In this case, the “pixel luminance” can be considered as an average value of luminance in the pixel group.

  Further, the above-described “brightness conversion coefficient” and “area-specific conversion coefficient” define the relationship between the luminance of the input pixel and the luminance of the output pixel. For example, the luminance of the output pixel determined for each luminance at the time of input. However, the present invention is not limited to this.

  In addition, the “pixel-specific conversion coefficient” described above defines the relationship between the luminance of the input pixel and the luminance of the output pixel, similarly to the “luminance conversion coefficient” and the “area-specific conversion coefficient”. Is given as a ratio between the luminance of the output pixel and the luminance of the input pixel determined for each luminance, but is not limited to this.

  As described above, with the image processing apparatus 1 according to the present embodiment, even if there is a portion related to luminance (or luminance range) that is infrequent when viewed from the entire image, good contrast is obtained including that portion. It becomes possible.

  However, in the image processing apparatus 1 according to the present embodiment, since the luminance conversion coefficient is calculated based on the luminance histogram for each area, an area in which a large number of pixels with low luminance values occupy a large number (that is, a dark area) is low. As the contrast of the luminance part (dark part) is increased, the noise of the image sensor that is likely to occur in a dark place becomes conspicuous. However, if noise filter processing is uniformly performed on the output image, the dark part While this noise can be removed, the sharpness of bright parts may be impaired.

  Therefore, in the following, a second embodiment of the image processing apparatus according to the present invention is proposed as means for solving the above problems, and the configuration and operation thereof will be described in detail.

  FIG. 8 is a block diagram showing a second embodiment of the image processing apparatus according to the present invention.

  As shown in the figure, the image processing apparatus 1-2 of the present embodiment has a configuration substantially similar to that of the first embodiment described above, and is characterized in that noise filter processing is performed only on low-luminance pixels. Yes. Therefore, the same parts as those of the first embodiment are denoted by the same reference numerals as those in FIG. 1, and the description thereof will be omitted. Hereinafter, the characteristic parts of this embodiment will be mainly described.

  The image processing apparatus 1-2 according to the present embodiment includes a noise filter processing unit 15 and a noise filter control unit 16 as means for realizing the noise filter processing.

  Of the circuit elements constituting the image processing apparatus 1-2, in addition to the above-described luminance conversion processing unit 11, luminance conversion coefficient calculation unit 12, and luminance conversion coefficient storage unit 13, a noise filter processing unit 15, and The noise filter control unit 16 is also preferably integrated in the semiconductor device.

  The noise filter processing unit 15 is means for applying a predetermined noise filter process to each pixel constituting the output image of the luminance conversion processing unit 11. As an example of the noise filter processing, the low-pass filter described above is referred to with reference to the luminance value of a certain pixel of interest (the pixel to be subjected to the noise filter processing) and the luminance values of its peripheral pixels (for example, four pixels on the top, bottom, left, and right). It is conceivable to perform a weighting operation similar to the filter processing.

  The noise filter control unit 16 is means for switching execution / non-execution of noise filter processing based on the luminance value for each pixel obtained by the luminance conversion coefficient calculation unit 12. Note that the noise filter control unit 16 of the present embodiment buffers the luminance value for each pixel obtained by the luminance conversion coefficient calculation unit 12 in the order of input in the luminance dynamic range correction processing (specifically, luminance histogram acquisition processing). In addition, an instruction is sent to the noise filter processing unit 16 so that only a pixel having a luminance value lower than a predetermined threshold is subjected to the noise filter processing.

  By adopting such a configuration, it is possible to determine whether or not noise filter processing is necessary for each pixel of the input image, and to switch between implementation / non-execution, so that the noise may be emphasized. While the noise filter processing is performed on the portion, it is possible to give priority to the sharpness of the bright portion without performing the noise filter processing in view of the fact that noise is not noticeable.

  Therefore, in the case of the image processing apparatus 1-2 of the present embodiment, when a desired output image is generated by performing luminance conversion processing (luminance dynamic range correction processing) on the input image, noise in dark portions is emphasized. However, it is possible to appropriately remove the noise in the dark part without impairing the sharpness of the bright part and improve the image quality and visibility of the output image.

  In the second embodiment, the case where the present invention is applied to an image processing apparatus that divides an input image into a plurality of areas and calculates a luminance conversion coefficient based on a luminance histogram for each area will be described as an example. However, as far as the application target of the present invention in which noise filter processing is performed only on low-luminance pixels, the present invention is not limited to the above embodiment, and the luminance conversion coefficient is calculated based on the luminance histogram of the entire input image. An image processing circuit that generates a desired output image by performing luminance conversion processing corresponding to a luminance conversion coefficient on each pixel constituting the input image, a semiconductor device formed by integrating the image processing circuit, and The present invention can be widely applied to all image processing apparatuses using the.

  The configuration of the present invention can be variously modified within the scope of the present invention in addition to the above embodiment.

  For example, in the second embodiment, the description has been given by taking as an example the configuration for switching between implementation / non-implementation of the noise filter processing using the noise filter control unit 16, but the configuration of the present invention is limited to this. For example, as a configuration for adjusting the characteristics of the noise filter processing based on the luminance value for each pixel obtained by the luminance conversion coefficient calculation unit 12 (for example, a configuration for variably controlling the coefficient of the weighting operation described above) Also good. By adopting such a configuration, it is possible to realize more flexible noise filter control as compared with a configuration in which execution / non-execution of noise filter processing is switched.

  Next, a third embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 9 is a block diagram showing a third embodiment of the image processing apparatus according to the present invention.

  As shown in this figure, the image processing apparatus 1-3 of the present embodiment has a configuration substantially similar to that of the first embodiment described above, and performs noise filter processing reflecting the correction amount of the luminance dynamic range correction processing. It is characterized by a point. Therefore, the same parts as those of the first embodiment are denoted by the same reference numerals as those in FIG. 1, and the description thereof will be omitted. Hereinafter, the characteristic parts of this embodiment will be mainly described.

  The image processing apparatus 1-3 according to the present embodiment includes a noise filter processing unit 15-2 and a filter coefficient setting unit 17 as means for realizing the noise filter processing.

  Of the circuit elements constituting the image processing apparatus 1-3, in addition to the above-described luminance conversion processing unit 11, luminance conversion coefficient calculation unit 12, and luminance conversion coefficient storage unit 13, a noise filter processing unit 15-2, Also, the filter coefficient setting unit 17 may be integrated in the semiconductor device.

  The noise filter processing unit 15-2 is a unit that performs predetermined noise removal processing on each pixel constituting the output image of the luminance conversion processing unit 11 based on the filter coefficient obtained by the filter coefficient setting unit 17. As an example of the noise removal process, the luminance value of the pixel that is the target of the noise removal process and the luminance value of the surrounding pixels (for example, the upper, lower, left, and right pixels) are referred to, and a predetermined weighting calculation is performed to obtain an image. It is conceivable to apply a low pass filter process spatially. By such noise removal processing, it is possible to reduce the luminance difference between adjacent pixels and make the noise inconspicuous.

  The filter coefficient setting unit 17 has a luminance value for each pixel constituting the input image (a luminance value before correction) and a luminance value for each pixel constituting the output image of the luminance conversion processing unit 11 (a luminance value after correction). ), And a filter coefficient for noise removal processing is set according to the calculation result.

  The image processing apparatus 1-3 configured as described above performs luminance conversion processing based on the luminance conversion coefficient on the input image obtained by moving image capturing, and displays this on the display. By performing such luminance conversion processing, it is possible to make the contrast of a portion of the luminance range with high frequency in the input image clearer.

  In addition, the image processing apparatus 1-3 according to the present embodiment is not configured to set the filter coefficient of the noise filter processing based on the luminance value (brightness) of the input image, and the luminance value before correction and the luminance value after correction The filter coefficient of the noise filter process is set based on the difference value (that is, the correction amount by the luminance conversion process).

  More specifically, in the image processing circuit 1-3 of the present embodiment, the filter coefficient setting unit 17 performs noise filter processing as the correction amount by the luminance conversion processing increases as schematically illustrated in FIG. In order to strengthen, the filter coefficient of the noise filter process is set with a linear or curved correlation. In FIG. 10, the target portions X1 to X4 of the luminance dynamic range correction process correspond to the target portions Y1 to Y4 of the noise filter processing, respectively.

  In this way, by adopting a configuration in which the noise filter processing is performed by reflecting the correction amount of the luminance dynamic range correction processing, a portion where the correction amount (the difference value) by the luminance conversion processing is large (the noise may be emphasized). The noise filter processing is applied to a portion with a small correction amount, and the noise filter processing is weakened for the portion with a small correction amount so that the sharpness can be given priority. Become. Therefore, in the case of the image processing apparatus 1-3 according to the present embodiment, when a desired output image is generated by performing luminance conversion processing (luminance dynamic range correction processing) on the input image, the sharpness of the output image is not impaired. It is possible to appropriately remove noise and improve the image quality and visibility of the output image.

  Next, a fourth embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 11 is a block diagram showing a fourth embodiment of the image processing apparatus according to the present invention.

  As shown in the figure, the image processing apparatus 1-4 according to the present embodiment has substantially the same configuration as that of the third embodiment described above, and an edge enhancement processing unit 18 is provided after the noise filter processing unit 15-2. In addition, the filter coefficient setting unit 17-2 for setting the filter coefficient of the edge enhancement processing unit 18 is provided instead of the filter coefficient setting unit 17 for setting the filter coefficient of the noise filter processing. have.

  The edge enhancement processing unit 18 is a means for performing predetermined edge enhancement processing on each pixel constituting the output image of the noise filter processing unit 15-2 based on the filter coefficient obtained by the filter coefficient setting unit 17-2. is there. As an example of the edge enhancement processing, the luminance value of the pixel to be subjected to the edge enhancement processing and the luminance value of the surrounding pixels (for example, the four pixels on the top, bottom, left, and right) are referred to and a predetermined weighting calculation is performed, thereby It is conceivable to apply a high-pass filter process spatially. By such edge enhancement processing, it is possible to make the outline of the output image obscured by the noise removal processing stand out and suppress the blur of the output image.

  The filter coefficient setting unit 17-2 includes a luminance value for each pixel constituting the input image (a luminance value before correction) and a luminance value for each pixel constituting the output image of the luminance conversion processing unit 11 (after the correction). This is a means for calculating a difference value (correction amount by luminance conversion processing) with respect to (luminance value) and setting filter coefficients for edge enhancement processing according to the calculation result.

  As described above, the image processing apparatus 1-4 according to the present embodiment determines the filter coefficient of the noise filter process based on the difference value (correction amount by the luminance conversion process) between the luminance value before correction and the luminance value after correction. Instead of setting, a filter coefficient for edge enhancement processing is set.

  More specifically, in the image processing circuit 1-4 of the present embodiment, the filter coefficient setting unit 17-2 performs edge enhancement as the correction amount by the luminance conversion process is smaller as schematically illustrated in FIG. In order to enhance the processing, the filter coefficient of the edge enhancement processing is set with a linear or curved correlation. In FIG. 12, the target portions X1 to X4 of the luminance dynamic range correction process correspond to the target portions Y1 to Y4 of the edge enhancement process, respectively.

  Thus, by adopting a configuration in which the edge enhancement process is performed by reflecting the correction amount of the luminance dynamic range correction process, a portion where the correction amount (the difference value) by the luminance conversion process is small (unnecessarily strong noise removal process is performed). For parts that may be applied), edge enhancement processing is applied more strongly, while for parts with a large amount of correction, edge enhancement processing is weakened and noise removal is prioritized over blurring of the output image. Is possible. Therefore, when the input image is subjected to luminance conversion processing (luminance dynamic range correction processing) to generate a desired output image, noise is appropriately removed without impairing the sharpness of the output image, and the image quality and visibility of the output image are reduced. Can be increased. Therefore, in the case of the image processing apparatus 1-4 according to the present embodiment, when a desired output image is generated by performing luminance conversion processing (luminance dynamic range correction processing) on the input image, the sharpness of the output image is not impaired. It is possible to appropriately remove noise and improve the image quality and visibility of the output image.

  Next, a fifth embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 13 is a block diagram showing a fifth embodiment of the image processing apparatus according to the present invention.

  As shown in the figure, the image processing apparatus 1-5 of the present embodiment has substantially the same configuration as that of the first embodiment described above, and cooperates with the image processing IC 20 and a control unit 30 (CPU [Central Processing Unit], etc.). It is characterized in that an image processing system configured to operate is constructed and brightness conversion processing reflecting the brightness of the input image is performed. Therefore, the same parts as those of the first embodiment are denoted by the same reference numerals as those in FIG. 1, and the description thereof will be omitted. Hereinafter, the characteristic parts of this embodiment will be mainly described.

  The image processing IC 20 calculates a luminance conversion coefficient based on the luminance histogram of the input image obtained by the imaging unit 10, and performs a luminance conversion process corresponding to the luminance conversion coefficient on each pixel constituting the input image. A semiconductor integrated circuit device that generates a desired output image, which includes the luminance conversion processing unit 11, the luminance conversion coefficient calculation unit 12, and the luminance conversion coefficient storage unit 13 described above, and relates to the luminance of the input image As a means for acquiring information and sending it out of the IC, a luminance detector 19 is integrated.

  The luminance detector 19 monitors the input image obtained by the imaging unit 10 to acquire a luminance histogram of the input image, and sends this to the control unit 30 as information regarding the luminance of the input image. When acquiring the luminance histogram described above, each pixel of the input image is divided into the upper 3 bits of the luminance information in order to speed up the processing, and the number of pixels belonging to each division (each luminance range). Should be counted.

  Also, in the image processing IC 20, the luminance conversion processing unit 11 performs a luminance conversion process that reflects information related to the luminance of the input image in accordance with an instruction from the control unit 30. The intensity of conversion processing is variably controlled.

  In addition, the luminance conversion processing unit 11 determines what ratio between the input luminance I and the output luminance O after the luminance conversion processing in accordance with an instruction from the control unit 30 in order to realize the intensity control of the luminance conversion processing described above. Is adjusted to adjust the parameter α that determines whether or not to output, and the final output brightness O ′ is determined. For example, as the value of the parameter α is larger, the luminance conversion process is more effective. Conversely, as the parameter α is smaller, the luminance conversion process is weaker.

  On the other hand, the control unit 30 receives information (luminance histogram) on the luminance of the input image sent from the luminance detection unit 19 of the image processing IC 20 and instructs the image processing IC 20 to perform luminance conversion processing reflecting this. It has a function to send.

  In this way, by constructing an image processing system that enables the luminance histogram of the captured image to be read using the control unit 30 and reflects the result on the parameter relating to the luminance conversion processing (intensity of the luminance conversion processing), Compared to the conventional configuration in which the parameters are fixedly set, luminance conversion processing suitable for the input image can be performed, so that the image quality and visibility of the output image can be improved.

  The image processing apparatus 1-5 of the present embodiment includes a liquid crystal display that displays an output image as the output unit 14, and the control unit 30 receives an input sent from the luminance detection unit 19 of the image processing IC 20. It is configured to receive information on the brightness of the image (brightness histogram) and send an instruction to the output unit 14 (particularly, a liquid crystal display driver that controls the driving current of the liquid crystal display) so as to perform backlight control reflecting this. Yes.

  With such a configuration, for example, if the input image obtained by the imaging unit 10 is bright, the surroundings are bright, and if the backlight brightness is not increased, the visibility of the output image is reduced. It becomes possible to increase the brightness of the light and make the output image easier to see. On the other hand, if the input image obtained by the image capturing unit 10 is dark, the brightness of the output image is reduced based on the judgment that the surrounding area is dark and the visibility of the output image is hardly lowered even if the backlight brightness is slightly reduced. Thus, power consumption can be reduced.

  That is, with the image processing apparatus 1-5 of the present embodiment, the brightness of the backlight can be feedback-controlled according to the brightness of the input image, so that the visibility of the output image and the efficiency of power consumption are improved. It is possible to achieve both.

  In the fifth embodiment, the case where the present invention is applied to an image processing apparatus that performs luminance conversion processing on an image of each frame obtained by moving image capturing has been described as an example. The scope of application of the invention is not limited to this, and is generally applied to image processing apparatuses that generate a desired output image by performing luminance conversion processing on an input image, such as an image processing apparatus that performs luminance conversion processing on a still image. It is possible to apply.

  For example, when the present invention is applied to an image playback apparatus that plays back content such as a movie, the visibility of the output image is reduced even if the brightness of the backlight is slightly reduced if the input image is bright with respect to the backlight control described above. It is possible to reduce power consumption by reducing the brightness of the backlight based on the judgment that it is difficult to do so. On the other hand, if the input image is dark, it is possible to make the output image easier to see by increasing the luminance of the backlight.

  Or, conversely, if the input image is bright, the backlight brightness is set to a normal value so that the brightness of the output image does not decrease, while if the input image is dark, the backlight brightness is The power consumption may be reduced by reducing the power consumption.

  In the fifth embodiment, the case where the present invention is applied to an image processing apparatus that divides an input image into a plurality of areas and calculates a luminance conversion coefficient based on a luminance histogram for each area will be described as an example. However, in terms of the application object of the present invention in which an image processing system is constructed by cooperating the image processing IC 20 and the control unit 30 and luminance conversion processing reflecting the luminance of the input image is performed, the above embodiment The luminance conversion coefficient is calculated based on the luminance histogram of the input image, and a luminance conversion process according to the luminance conversion coefficient is performed on each pixel constituting the input image, thereby obtaining a desired value. The present invention can be widely applied to all image processing apparatuses that generate output images.

  Further, in terms of performing backlight control of the liquid crystal display based on the luminance information of the input image, in addition to the above configuration mainly including the control unit 30, the image processing IC 20 and the output unit 14 (particularly, a liquid crystal display driver). It is also possible to adopt a configuration mainly composed of In addition to the above-described configuration built in the image processing IC 20, the luminance detection unit 19 may be configured as a separate independent IC or a configuration built in the output unit 14.

  Hereinafter, a specific configuration for realizing the above backlight control will be described in detail.

  FIG. 14 is a block diagram showing a configuration example of the backlight driving device according to the present invention.

  As shown in the figure, the backlight driving device of this configuration example reflects the detection result of the luminance detection unit 100 and the luminance detection unit 100 that acquires information related to the luminance of the input image displayed on the liquid crystal display 300. And a drive unit 200 that performs backlight control of the display 300.

  In the backlight drive device of this configuration example, the luminance detection unit 100 divides each pixel of the input image into a plurality of luminance ranges, and counts the number of pixels (HYS_count0 to HYS_count7) belonging to each division to input The brightness histogram of the image is acquired.

  FIG. 15 is a diagram illustrating an example of a luminance histogram of an input image. Note that the horizontal axis in FIG. 15 indicates luminance (Y), and the vertical axis indicates the number of pixels (HYS_count).

  In the backlight driving apparatus of this configuration example, the driving unit 200 includes a first adder 201, a multiplier 202, a second adder 203, and a duty determination unit 204.

  The first adder 201 is means for adding the count values (HYS_count0 to HYS_count7) for each section obtained by the luminance detection unit 100. That is, the first adder 201 obtains the total number of pixels (HYS_TOTAL) for one frame of the input image.

  The multiplier 202 is means for multiplying the count values (HYS_count0 to HYS_count7) for each section obtained by the luminance detection unit 100 by a predetermined coefficient (PWM_HCOEF0 to PWM_HCOEF7).

  For the above coefficients (PWM_HCOEF0 to PWM_HCOEF7), an appropriate weighting is applied to the count value for each section, and finally any value can be set as long as the luminance value for one frame of the input image can be quantified. It doesn't matter. For example, if priority is given to simplifying the hardware configuration, PWM_HCOEF0 = 0, PWM_HCOEF1 = 1/64, PWM_HCOEF2 = 1/32, PWM_HCOEF3 = 1/16, PWM_HCOEF4 = 1/8, PWM_HCOEF5 = 1/4, PWM_HCOEF6 The above coefficients (PWM_HCOEF0 to PWM_HCOEF7) may be set so that each time the luminance range falls by one division, such as = 1/2 and PWM_HCOEF7 = 1, the weighting is reduced to one half. It is also possible to multiply the count values of different sections by the same coefficient.

  The second adder 203 is a means for adding the count values for each section multiplied by a predetermined coefficient (PWM_HCOEF0 to PWM_HCOEF7) by the multiplier 202. That is, in the second adder 202, a distribution sum (hys_total) necessary for determining the duty of the PWM signal is obtained.

  The duty determination unit 204 determines the duty of the PWM signal used for backlight control of the liquid crystal display 300 based on the calculation result (HYS_TOTAL) of the first adder 201 and the calculation result (hys_total) of the second adder 203. Means.

  More specifically, the duty determination unit 204 normalizes the calculation result (hys_total) of the second adder 203 with the calculation result (HYS_TOTAL) of the first adder 201, and calculates the calculation result (dutyA) in a table. It is configured to determine the duty (PWM_DUTY) of the PWM signal by converting.

  FIG. 16 is a diagram illustrating an example of a conversion table used in the duty determination unit 204. Note that the horizontal axis of FIG. 16 indicates the reference duty (dutyA) obtained by the normalization process, and the vertical axis indicates the duty of the PWM signal (PWM_DUTY).

  By using the above conversion table, it is possible to arbitrarily set the correlation between the reference duty (dutyA) and the duty of the PWM signal (PWM_DUTY), and thus the high level width (H width) of the PWM signal. Become. If linearity (linearity) is provided between the reference duty (dutyA) and the PWM signal duty (PWM_DUTY), the reference duty (dutyA) is used as it is as the PWM signal duty (PWM_DUTY). It will be.

  Further, as the above conversion table, the duty (PWM_DUTY) of the PWM signal with respect to the reference duty (dutyA) may be set discretely, and linear interpolation may be performed between the discrete values. With this configuration, the data amount of the conversion table can be reduced.

  The conversion table is preferably stored in a register or the like so that the user can rewrite it arbitrarily.

  The backlight of the liquid crystal display 300 is driven and controlled (current control) based on the PWM signal.

  FIG. 17 is a diagram for explaining the PWM drive of the backlight.

  As shown in this figure, the PWM signal is a pulse signal that alternately repeats a high level period (on period) and a low level period (off period) in a predetermined cycle, and variably controls its duty (high level width). This makes it possible to adjust the average value of the drive current that flows through the backlight. That is, the backlight becomes brighter as the duty of the PWM signal is increased, and the backlight is darker as the duty of the PWM signal is decreased.

  For example, when the conversion table shown in FIG. 16 is used, when the input image is bright and the reference duty (dutyA) is large, the backlight brightness is increased and the visibility is maintained, while the input image is dark. When the reference duty (dutyA) is small, it is possible to reduce the power consumption by reducing the luminance of the backlight.

  Next, a sixth embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 18 is a block diagram showing a sixth embodiment of the image processing apparatus according to the present invention.

  As shown in FIG. 18, the image processing apparatus according to the present embodiment includes an image generation apparatus 10-2, an image processing IC 20-2, a display apparatus 14-2, and an illuminance sensor 40.

  The image generation device 10-2 is a unit that generates a color input image IN, and any one of an imaging device, a broadcast reception device, and a media playback device can be used.

  As the above imaging device, a digital video camera that performs moving image shooting, a digital still camera that performs still image shooting, and the like can be used. As the image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) sensor or a CCD is used. A [Charge Coupled Devices] sensor or the like can be used. Thus, when an imaging device is used as the image generation device 10-2, the input image IN is a captured image obtained by the imaging device.

  Also, as the broadcast receiving apparatus, a digital television broadcast receiving apparatus or the like can be used. Thus, when a broadcast receiving device is used as the image generating device 10-2, the input image IN is a received image obtained by the broadcast receiving device.

  In addition to the above-mentioned media playback devices such as video CDs, DVDs, Blu-ray discs, hard disks, semiconductor memories, and other drive devices with playback functions, personal computers with playback functions of video content provided via the Internet. A computer or the like can be used. In this way, when a media playback device is used as the image generation device 10-2, the input image IN is a playback image obtained by the media playback device.

  The image processing IC 20-2 is a semiconductor device in which the image processing unit 21 and the color correction control unit 22 are integrated.

  The image processing unit 21 includes a color correction unit 21a that performs a predetermined color correction process on the input image IN. Although not clearly shown in FIG. 1, the image processing unit 21 performs various image processing (luminance dynamic range correction, backlight correction, etc.) on the input image IN in addition to the color correction processing unit 21a. Includes circuit blocks.

  The color correction control unit 22 is means for controlling the intensity of the color correction process based on the detection signal DET obtained by the illuminance sensor 40.

  The display device 14-2 includes a display such as an LCD [Liquid Crystal Display], and is a means for displaying an output image OUT that has been subjected to predetermined image processing by the image processing IC2.

  The illuminance sensor 40 is a means for generating a detection signal DET corresponding to the illuminance of ambient light and sending it to the color correction control unit 22 of the image processing IC 20-2. Note that the illuminance sensor 40 is desirably arranged as close to the display device 14-2 as possible in order to accurately measure the illuminance of the ambient light incident on the display device 14-2. A photodiode or a phototransistor can be used as the light receiving element of the illuminance sensor 40.

  The operation (particularly color correction control) of the image processing apparatus having the above configuration will be described in detail.

  When performing color correction processing on the input image IN using the image processing IC 20-2, the color correction control unit 22 increases the color correction processing as the ambient light illuminance increases based on the detection signal DET of the illuminance sensor 40. The color correction unit 21a is controlled (so as to enhance the color component of the input image IN).

  By performing such color correction control, even if the illuminance of the ambient light is high and the color reproducibility of the display device 14-2 is lost, the color correction processing for the input image IN is strengthened to compensate for this. Therefore, the color of the output image OUT does not appear light, and the visibility of the output image OUT can be improved.

  On the other hand, when performing color correction processing on the input image IN, the color correction control unit 22 controls the color correction unit 21a based on the detection signal DET of the illuminance sensor 40 so that the color correction processing is weakened as the illuminance of the ambient light decreases. To do.

  By performing such color correction control, when the illuminance of ambient light is low, the color correction processing for the input image IN is weakened, so that the hue of the output image OUT does not become unnatural, and the output image It becomes possible to improve the visibility of OUT.

  Next, detailed description will be given of the brightness correction processing of the input image in the image processing unit 21.

  FIG. 19 is a block diagram illustrating a configuration example of the image processing unit 21.

  As shown in the figure, the image processing unit 21 includes a luminance correction unit 211, a color correction unit 212, a luminance histogram acquisition unit 213, and a correction coefficient calculation unit 214.

  The luminance correction unit 211 is a unit that performs a predetermined correction process (luminance dynamic range correction process) on the luminance component Y of the input image IN.

  The color correction unit 212 is a unit that performs a predetermined correction process on the color difference components U (= BY) and V (= RY) of the input image IN, and corresponds to the color correction unit 21a of FIG. It is a block.

  The luminance histogram acquisition unit 213 is a means for acquiring a luminance histogram (frequency of each luminance range, that is, distribution of the number of pixels belonging to each predetermined luminance range) of the input image IN.

  The correction coefficient calculation unit 214 calculates the luminance correction coefficient α used by the luminance correction unit 211 and the color correction coefficient β used by the color correction unit 212 based on the luminance histogram obtained by the luminance histogram acquisition unit 213. It is.

  The calculation of the brightness correction coefficient α will be described. Based on the brightness histogram of the input image IN, the correction coefficient calculation unit 214 increases the frequency of the output image OUT (brightness) for each brightness range in the input image IN (brightness component Y). The luminance correction coefficient α is calculated so that a wider luminance range is assigned in the component Y ′), and conversely, the lower the frequency is, the narrower luminance range is assigned in the output image OUT (luminance component Y ′). Accordingly, the luminance correction coefficient α is a separate value for each luminance or for each predetermined luminance range.

  As a result, the contrast (brightness difference) becomes more ambiguous in the portion of the luminance range that is less frequent in the input image IN, but conversely, the portion of the luminance range that is frequent in the input image IN is more contrasted. Becomes clear. Therefore, when viewed as an entire image, the contrast can be made clearer than at the time of input.

  On the other hand, the calculation of the color correction coefficient β will be described. The correction coefficient calculation unit 214 decreases the color correction amount for a portion with a low luminance value and a large luminance correction amount (a portion with a large number of pixels with low luminance values). Thus, the color correction coefficient β is calculated.

  For example, as shown in FIG. 20, the correction coefficient calculation unit 214 includes a first correction table 214a for a bright input image and a second correction table 214b for a dark input image. If the input image IN is brighter than the predetermined threshold based on the luminance histogram, the color correction coefficient β is calculated using the first correction table 214a. If the input image IN is darker than the predetermined threshold, the color correction amount is calculated. The color correction coefficient β may be calculated by using the second correction table 214b so as to reduce the value.

  Alternatively, as shown in FIG. 21, the correction coefficient calculation unit 214 includes a reference correction table 214c and an adjustment coefficient calculation unit 214d, and calculates the reference correction coefficient γ using the reference correction table 214c. Alternatively, the color correction coefficient β (= γ × δ) may be calculated by multiplying this by the adjustment coefficient δ corresponding to the luminance histogram of the input image IN.

  Thereby, when the brightness correction processing of the input image IN is composed of the brightness correction processing and the color correction processing, when correcting the dark portion of the input image IN brightly by the brightness correction processing, the color correction amount is suppressed to be small, Since noise (unnecessary chroma components) can be removed, it is possible to obtain a natural output image OUT, and it is possible to improve the visibility of the output image OUT without depending on the illuminance of ambient light. It becomes.

  In the sixth embodiment, the configuration in which the detection signal DET of the illuminance sensor 40 is directly input to the color correction control unit 22 of the image processing IC 20-2 has been described as an example. The configuration is not limited to this, and may be configured to be indirectly input via a CPU [Central Processing Unit] or the like.

  Next, a seventh embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 22 is a block diagram showing a seventh embodiment of the image processing apparatus according to the present invention. As shown in this figure, the image processing apparatus of this embodiment includes a camera module 10-3, an image processing IC 20-3, and a control unit (CPU, DSP, etc.) 30-3.

  FIG. 23 is a block diagram illustrating a configuration example of the image processing IC 20-3. As shown in the figure, the image processing IC 20-3 of this configuration example includes a color correction unit A1, a luminance determination unit A2, an image enhancement unit A3, a register A4, an I2C interface unit A5, an edge enhancement / It includes a gamma correction unit A6, a PWM signal generation control unit A7, and a timing generation unit A8.

  The image processing IC 20-3 receives an input signal (image data signal CAMDI [7: 0]) from the camera module 10-3 based on a control signal (data signal SDA, clock signal SDC) serially input from the control unit 30-3. ], The horizontal synchronization signal CAMHSI, the vertical synchronization signal CAMVSI, and the clock signal CAMMCI) are subjected to predetermined arithmetic processing (luminance dynamic range correction, color correction, etc.), and a desired output signal (image data signal CAMDO [7: 0], horizontal Synchronization signal CAMHSO, vertical synchronization signal CAMVSO, and clock signal CAMCKO), and outputs these to the control unit 30-3. Note that examples of the application target of the image processing IC 20-3 include a security device and an in-vehicle camera / monitor.

  The characteristics of the image processing IC 20-3 will be described. The first feature is that it corresponds to image data from a QCIF size (176 × 144 pixels) to a maximum WVGA + size (886 × 480 pixels). The second feature is that the input / output data format corresponds to “ITU-R BT.656-4” or “YCbCr with synchronization signal”. A third feature is that an image enhancement mode, a through mode, and a sleep mode are provided as operation modes. A fourth feature is that register setting can be performed via a two-wire serial interface (I2C interface unit A5). A fifth feature is that a function unit (PWM generation control unit A7) that outputs a PWM signal for controlling the LCD backlight in conformity with an image is provided. A sixth feature is that a function unit (edge enhancement / gamma correction unit A6) for applying an edge enhancement filter or a gamma filter to the output image is incorporated.

  A functional overview of the image processing IC 20-3 will be described. The image processing IC 20-3 is an adaptive image enhancer for the camera interface, and is used by being inserted into the camera interface as shown in FIG. The image processing IC 20-3 corrects the dark portion of the input image to a bright image and the excessively bright portion by reducing the brightness by the luminance dynamic range correction processing using the luminance determination unit A2 and the image enhancement unit A3. It has a function to output. The image processing IC 20-3 also has a function of correcting an input image to a more vivid image by color correction processing using the color correction unit A1. Note that the above-described image processing is performed by analyzing the previous frame and reflecting it in the current frame.

  The image processing IC 20-3 is compatible with YUV format data input, and has a function of outputting image-corrected data in the same format. The register value of the register A4 built in the image processing IC 20-3 is set through the I2C interface unit A5. The image processing IC 20-3 operates based on the clock signal CAMCKI input from the camera module 10-3. Further, the image processing IC 20-3 performs predetermined timing adjustment processing on the horizontal synchronization signal CAMHSI, the vertical synchronization signal CAMVSI, and the clock signal CAMCKI input from the camera module 10-3, and these are subjected to the horizontal synchronization signal CAMHSO and the vertical synchronization signal. As a synchronization signal CAMVSO and a clock signal CAMCKO, a functional unit (timing generation unit A8) for sending to the control unit 30-3 is provided. In addition, the image processing IC 20-3 receives an input device (such as a camera or a video reproduction device) or an output device (such as a CPU or DSP) that is externally connected to the image processing IC 20-3 in response to a mode selection signal MSEL0 / 1/2 that is input from the outside. ) Is also provided with a function for selecting a suitable operation mode.

  24 and 25 are both terminal function tables of the image processing IC 20-3, and pin numbers, terminal names, input / output types (In / Out), active levels, pin states during reset (Init), and Function description is described. Note that “*” in the active level column indicates that the setting can be arbitrarily changed using the register A4. “* 1” indicates a terminal that is suspended during reset (initial state). “* 2” and “* 3” indicate terminals to be connected to GND. “* 4” indicates a terminal to be connected to VDDIO (digital IO power supply). “* 5” indicates a terminal to be opened.

  Next, an outline of the YUV interface timing will be described with reference to FIGS.

  FIG. 26 is a timing chart showing the “ITU-R BT.656” input format (horizontal direction bottom). FIG. 27 is a timing chart showing the “ITU-R BT.656” input format (vertical direction) in NTSC. FIG. 28 is a timing chart showing an “ITU-R BT.656” input format (vertical bottom) in PAL. Note that the frequency of the clock signal CAMCKI should not be changed during operation.

  FIG. 29 is a timing chart showing the “YCbCr with synchronization signal” 8-bit input format (horizontal direction). In the figure, “YUV_XST”, “XSIZE × 2”, “AIE_XST × 2”, and “AIE_XSIZE × 2” are all register values that can be set using the register A4. In addition, the polarities of the clock signal CAMCKI, the vertical synchronization signal CAMVSI, and the horizontal synchronization signal CAMHSI can be individually set by changing the value of the POL register (index address: E1h). FIG. 29 shows the case where data is captured at the falling edge of the clock signal CAMCKI (CKPOL = 1), and the polarity of the horizontal synchronization signal CAMHSI is low active (HSPOL = 0). ing. Also, it should be noted that the horizontal synchronization signal CAMHSI does not become low level except in the sync period (interval in which CAMHSI is low level in FIG. 29). Further, the frequency of the clock signal CAMCKI should not be changed while the data signals CAMDI0 to CAMDI7 are being input.

  FIG. 30 is a timing chart showing the “YCbCr with synchronization signal” 8-bit input format (vertical direction). “YSIZE”, “AIE_YST”, and “AIE_YSIZE” in the figure are register values that can be set using the register A4. FIG. 30 shows a case where the polarities of the vertical synchronization signal CAMVSI and the horizontal synchronization signal CAMHSI are both low active (VSPOL = 0, HSPOL = 0).

  Next, an outline of the I2C interface will be described with reference to FIG.

  FIG. 31 is a diagram showing an I2C interface format. In addition, the code | symbol S in a figure has shown the start condition, and the code | symbol P has shown the stop condition. Reference symbol A (S) indicates an acknowledge by the slave, and reference symbol A (M) indicates an acknowledge by the master. A (S) bar indicates non-acknowledgement by the slave, and A (M) bar indicates non-acknowledgement by the master. The slave address is 42h when I2CDEV0 = 0, and 43h when I2CDEV0 = 1. The sub address is automatically incremented when two or more consecutive accesses are performed for both writing and reading.

  Next, the setting of the register A4 will be described in detail.

  FIG. 32 is a map showing the contents of the initialization register SRST. The initialization register SRST (index address: FFh) is a register value related to software reset, and includes a 1-bit parameter SRST (bit number 0, write attribute). The initial value of the parameter SRST is “0”, and writing “1” to the parameter SRST causes the image processing IC 20-3 to be software reset. After the software reset, the initialization register SRST should not be accessed for 100 [ns]. There is no need to perform a software reset after a hardware reset.

  FIG. 33 is a map showing the contents of the parameter update register PARAMSET. The parameter update register PARAMSET (index address: 11h) is a register value related to parameter update. In addition to the 1-bit parameter PARAM_SET (bit number 0, read / write attribute), the reserved bits RESERVED (bit number 1, 2). The initial value of the parameter PARAM_SET is “0”, and by writing “1” to the parameter PARAM_SET, the parameters of various register values are updated. More specifically, the parameter update of various register values is performed at the head of the frame after the parameter update register PARAMSET is written. The parameter update register PARAMSET is automatically cleared after the parameters of various register values are updated.

  FIG. 34 is a map showing the contents of the mode register MODE. The mode register MODE (index address: 12h) is a register value related to setting of various operation modes of the image processing IC 20-3, and includes a 2-bit parameter MODE (bit number 0 to 1, read / write attribute) and a 1-bit parameter. Parameter TH_TYPE (bit number 2, read / write attribute) and 1-bit parameter SUSP (bit number 3, read / write attribute) are included.

  The parameter MODE is a parameter for switching between the sleep mode, the through mode, and the enhancement mode, and its initial value is “00b”. By writing this value, the operation mode becomes the sleep mode. In the sleep mode, the image processing IC 20-3 enters a sleep state. By writing “01b” in the parameter MODE, the operation mode becomes the through mode. In the through mode, image correction is not performed on the input data, and the input data becomes output data as it is. Note that the parameter TH_TYPE must be set to “1” before the operation mode is set to the through mode. By writing “11b” in the parameter MODE, the operation mode becomes the enhancement mode. In the enhanced mode, the result of performing image correction on input data is output as output data. Note that writing “10b” to the parameter MODE is prohibited. FIG. 35 is a logical table showing the relationship between the parameter MODE and the operation mode.

  The parameter TH_TYPE is a parameter for setting the operation type of the through mode, and its initial value is “0”. By writing “1” in the parameter TH_TYPE, the image processing IC 20-3 delays the data and the synchronization signal input from the camera module 10-3 in the through mode by the same number of cycles as in the enhancement mode, and outputs the delayed data. . Note that writing “0” to the parameter TH_TYPE is prohibited.

  The parameter SUSP is a parameter for setting the suspend mode of I / O [Input / Output], and its initial value is “1”. Writing “0” to the parameter SUSP cancels the suspend mode, and writing “1” to the parameter SUSP sets the suspend mode.

  FIG. 36 is a map showing the contents of the camera X direction pixel size setting register DXSIZE. The camera X direction pixel size setting register DXSIZE (index address: 14h and 15h) is a register value for setting the number of pixels in the X direction of input image data, and is an 11-bit parameter XSIZE (bit number 0 to address 14h). 7 and bit numbers 0 to 2 of the address 15h, read / write attributes). The initial value of the parameter XSIZE is “010 1000 0000b” (640 pixels), and can be set from 144 pixels to 1024 pixels.

  FIG. 37 is a map showing the contents of the camera Y direction pixel size setting register DYSIZE. The camera Y direction pixel size setting register DYSIZE (index address: 16h and 17h) is a register value for setting the number of pixels in the Y direction of the input image data, and is an 11-bit parameter YSIZE (bit numbers 0 to 16 of the address 16h). 7 and bit numbers 0 to 2 of the address 17h, read / write attributes). The initial value of the parameter YSIZE is “001 1110 0000b” (480 pixels).

  FIG. 38 is a map showing the contents of the image correction effective area X direction start position setting register AIEXST. The image correction effective area X-direction start position setting register AIEXST (index address: 18h and 19h) is a register value for setting the X-direction start position of the image correction effective area, and is a 10-bit parameter AIEXST (bit of address 18h). Number 0-7, bit number 0-1 of address 19h, read / write attribute). The initial value of the parameter AIEXST is “00 0000 0000b”, and can be set from 0 to (XSIZE−AIEXSIZE).

  FIG. 39 is a map showing the contents of the image correction effective area Y-direction start position setting register AIEYST. The image correction effective area Y-direction start position setting register AIEYST (index addresses: 1Ah and 1Bh) is a register value for setting the Y-direction start position of the image correction effective area, and is a 10-bit parameter AIEYST (bit of address 1Ah). Number 0 to 7 and bit number 0 to 1 of address 1Bh, read / write attribute). The initial value of the parameter AIEYST is “00 0000 0000b”, and can be set from 0 to (YSIZE−AIEYSIZE).

  FIG. 40 is a map showing the contents of the image correction effective area X-direction size setting register AIEXSIZE. The image correction effective area X-direction size setting register AIEXSIZE (index address: 1Ch and 1Dh) is a register value for setting the number of pixels in the X direction of the image correction effective area, and is an 11-bit parameter AIEXSIZE (address 1Ch bit). Number 0 to 7 and bit number 0 to 2 of address 1Dh, read / write attribute). The initial value of the parameter AIEXST is “010 1000 0000b” (640 pixels), and can be set from 144 to XSIZE.

  FIG. 41 is a map showing the contents of the image correction effective area Y-direction size setting register AIEYSIZE. The image correction effective area Y-direction size setting register AIEYSIZE (index address: 1Eh and 1Fh) is a register value for setting the number of pixels in the Y direction of the image correction effective area, and is an 11-bit parameter AIEYSIZE (address 1Eh bit). Number 0-7, bit number 0-2 of address 1Fh, read / write attribute). The initial value of the parameter AIEYST is “001 1110 0000b” (480 pixels), and can be set from 144 to YSIZE.

  FIG. 42 is a diagram schematically showing an image correction effective area of input image data.

  As shown in FIG. 42, the image processing IC 20-3 according to the present embodiment has a function of performing image correction processing only on an arbitrary image correction effective area, not the entire input image as a correction target. . That is, the luminance determination unit A2 (corresponding to the luminance conversion coefficient calculation unit 12 in other embodiments) divides only the image correction effective region shown in FIG. The image enhancement unit A3 (corresponding to the luminance conversion processing unit 11 in other embodiments) corresponds to the luminance conversion coefficient for each area with respect to the pixels constituting each area. The luminance conversion process is performed. Note that the luminance conversion coefficient calculation process and the luminance conversion process are the same as those in the first embodiment described above, and a duplicate description is omitted.

  As described above, the image processing IC 20-3 of the present embodiment includes the correction effective area setting unit (in the example of FIG. 23, the register A4 and the I2C interface unit A5) that sets the correction effective area of the input image; A luminance conversion coefficient calculation unit (in the example of FIG. 23, luminance determination unit A2) that calculates a luminance conversion coefficient for each area after dividing only the effective correction area into a plurality of areas; A luminance conversion processing unit (image enhancement unit A3 in the example of FIG. 23) that performs a luminance conversion process according to the luminance conversion coefficient for each area on the constituent pixels. . With this configuration, it is possible to improve the image quality and visibility of an arbitrary area included in the input image as intended by the user.

  FIG. 43 is a map showing the contents of the brightness correction strength setting register STRENGTH. The luminance correction strength setting register STRENGTH (index address: 21h) is a register value for setting the strength of image correction (luminance conversion processing strength), and is a 7-bit parameter STR (bit numbers 0 to 6, read / write). Attribute). The initial value of the parameter STR is “110 0000b” (96d), and can be set from 0 to 127d. Note that the configuration for variably controlling the intensity of the luminance conversion process is the same as that of the fifth embodiment described above, and therefore a duplicate description is omitted.

  FIG. 44 is a map showing the contents of the color difference correction intensity setting register UV_STRENGTH. The color difference correction intensity setting register UV_STRENGTH (index address 22h) is a register value for setting the intensity of color difference correction (intensity of color correction processing), and is a 4-bit parameter UV_STR (bit number 0 to 3, read / write attribute) ) And a 4-bit parameter V_ENHANCE (bit numbers 4 to 7, read / write attribute).

  The initial value of the parameter UV_STR is “0110b” (6d), and can be set from 0 to 13d. Note that the larger the value of the parameter UV_STR, the stronger the color development. Also, writing “1110”, “1111” (14d, 15d) to the parameter UV_STR is prohibited.

  The initial value of the parameter V_ENHANCE is “0000b” and can be set from 0 to 15d. Note that redness increases as the value of the parameter V_ENHANCE increases. Specifically, redness is emphasized by about +3 [%] × set value.

  FIG. 45 is a map showing the contents of the noise suppression setting register NOISE_SUP. The noise suppression setting register NOISE_SUP (index address: 23h) is a register value for setting a noise suppression value, and includes a 7-bit parameter NOISE_SUP (bit numbers 0 to 6, read / write attributes). The initial value of the parameter NOISE_SUP is “110 0000b” (96d), and can be set from 10d to 127d. Note that noise is suppressed as the value of the parameter NOISE_SUP increases. In addition, it is prohibited to write “0000” to “1001” (0 to 9d) in the parameter NOISE_SUP. The configuration for variably controlling the noise suppression value is the same as that of the third embodiment described above, and thus a duplicate description is omitted.

  FIG. 46 is a map showing the contents of the edge enhancement filter setting register EDG_CNT. The edge emphasis filter setting register EDG_CNT (index address: 24h) is a register value for controlling the operation of the edge emphasis filter, and includes a 4-bit parameter EDG_ST (bit number 0 to 3, read / write attribute) and 1 bit. Parameter EDG_EN (bit number 7, read / write attribute).

  The initial value of the parameter EDG_ST is “0000b” and can be set from 0 to 15d. Note that edge enhancement becomes stronger as the value of the parameter EDG_ST increases.

  The parameter EDG_EN is a parameter for setting enable of the edge enhancement filter, and its initial value is “0”. Writing “0” to the parameter EDG_EN disables the edge enhancement filter, and writing “1” to the parameter EDG_EN enables the edge enhancement filter. The edge enhancement filter is effective only in the ITU656 format, and in other cases, the parameter EDG_EN should be set to “0”.

  Note that the configuration for variably controlling the strength of the edge enhancement filter is the same as that in the fourth embodiment described above, and therefore a duplicate description is omitted.

  FIG. 47 is a map showing the contents of the response time setting register RESP_SET. The response time setting register RESP_SET (index address: 25h) is a register value for setting the response time of the image correction effect, and includes a 4-bit parameter RESP_SET (bit number 0 to 3, read / write attribute). Yes. The initial value of the parameter RESP_SET is “0000b” and can be set from 0 to 15d. The response time becomes longer as the value of the parameter RESP_SET is larger.

  FIG. 48 is a map showing the contents of the post filter enable register PFLT_EN. The post filter enable register PFLT_EN (index address 26h) is a register value for setting the output gamma filter enable, and includes a 1-bit parameter PFLT_EN (bit number 0, read / write attribute). The initial value of the parameter PFLT_EN is “0”. The output gamma filter is disabled by writing “0” to the parameter PFLT_EN, and the output gamma filter is enabled by writing “1” to the parameter PFLT_EN. When the output gamma filter is enabled, the gamma filter is applied to the image after image correction using the characteristics set in the post filter characteristics register (POFLT0 to POFLT8 described later).

  FIG. 49 is a map showing the contents of the output gamma characteristic registers POFLT0 to POFLT8. The output gamma characteristic registers POFLT0 to POFLT8 (index addresses: 27h to 2Fh) are all register values for setting the output gamma characteristic (see FIG. 50). Each of the 8-bit parameters POFLT0 to POFLT8 (bit number 0) ~ 7, read / write attributes). The initial value of the parameter POFLT0 is “0000 0000b”. The initial value of the parameter POFLT1 is “0001 1111b” (31d). The initial value of the parameter POFLT2 is “0011 1111b” (63d). The initial value of the parameter POFLT3 is “0101 1111b” (95d). The initial value of the parameter POFLT4 is “0111 1111b” (127d). The initial value of the parameter POFLT5 is “1001 1111b” (159d). The initial value of the parameter POFLT6 is “1011 1111b” (191d). The initial value of the parameter POFLT7 is “1101 1111b” (223d). The initial value of the parameter POFLT8 is “1111 1111b” (255d).

  FIG. 51 is a map showing the contents of the luminance distribution setting register LFREQSET. The luminance distribution setting register LFREQSET (index address: 30h) is a register value related to the luminance value frequency fetch operation, and is a 1-bit parameter LFREQST (bit number 0, read / write attribute) and a 1-bit parameter LFREQPOS (bit number). 1, read / write attributes).

  When “1” is written in the parameter LFREQST, the frequency of the luminance value of the next frame image is captured. Note that while the value of the parameter LFREQST is maintained at “1”, the process of fetching the frequency of the brightness value is in an incomplete state, and the brightness distribution registers LFREQ0 to LFREQ and the brightness distribution parameter register LFREQ_TOTAL which will be described later. This data is invalid. On the other hand, if the value of the parameter LFREQST returns to “0”, it means that the process of fetching the luminance value frequency has been completed, and the data in the luminance distribution registers LFREQ0 to LFREQ and the luminance distribution parameter register LFREQ_TOTAL are valid. It becomes. Therefore, after writing “1” in the parameter LFREQST and returning the value to “0”, the luminance distribution registers LFREQ0 to LFREQ and the luminance distribution parameter register LFREQ_TOTAL can be read.

  The parameter LFREQPOS is a parameter for selecting an image from which the luminance value frequency is taken, and its initial value is “0”. Writing “1” to the parameter LFREQPOS takes in the luminance value of the output image (image corrected image), and writing “0” into the parameter FREQPOS takes in the luminance value of the input image. .

  FIG. 52 is a map showing the contents of the luminance distribution registers LFREQ0 to LFREQ7. The luminance distribution registers LFREQ0 to LFREQ7 (index addresses 31h to 38h) are all register values for storing the frequency of the luminance value of the image. Each of the 8-bit parameters LFREQ0 to LFREQ7 (bit numbers 0 to 7, read attribute) ) Is included. The parameter LFREQ0 stores the number of pixels having luminance values 0 to 31. The parameter LFREQ1 stores the number of pixels having luminance values 32 to 63. The parameter LFREQ2 stores the number of pixels having luminance values of 64 to 95. The parameter LFREQ3 stores the number of pixels having luminance values of 96 to 127. The parameter LFREQ4 stores the number of pixels having luminance values 128 to 159. The parameter LFREQ5 stores the number of pixels having luminance values 160 to 191. The parameter LFREQ6 stores the number of pixels having luminance values 192 to 223. The parameter LFREQ7 stores the number of pixels having luminance values 224 to 255.

  FIG. 53 is a map showing the contents of the luminance distribution parameter register LFREQ_TOTAL. The luminance distribution parameter register LFREQ_TOTAL (index address: 39h) is a register value for storing the luminance distribution parameter (total number of pixels), and includes an 8-bit parameter LFREQ_TOTAL (bit number 0 to 7, read attribute). It is out. Note that the luminance distribution parameter register operates only at the time of analysis or enhancement.

  Note that since the luminance distribution capturing operation is the same as that of the fifth embodiment described above, a duplicate description is omitted.

  FIG. 54 is a map showing the contents of the PWM control register. The PWM control register PWMCNT (index address: 40h) is a register value for controlling the output of the PWM signal, and includes a 2-bit parameter PWM_SEL (bit number 0 to 1, read / write attribute) and a 1-bit parameter. PWM_HOST (bit number 6, read / write attribute) and 1-bit parameter PWM_EN (bit number 7, read / write attribute) are included.

  The parameter PWM_SEL is a parameter for setting the output terminal of the PWM signal, and its initial value is “00b”. Writing "00b" to the parameter PWM_SEL makes the PWM signal non-output state, and writing "01b" to the parameter PWM_SEL makes the PWM signal output via the PWMO terminal. Note that writing “10b” and “11b” to the parameter PWM_SEL is prohibited.

  The parameter PWM_HOST is a parameter for setting the duty control method of the PWM signal, and its initial value is “0”. By writing “0” in the parameter PWM_HOST, the image processing IC 20-3 automatically controls the duty of the PWM signal. By writing “1” to the parameter PWM_HOST, the duty of the PWM signal is controlled externally. At this time, a register value of a PWM DUTY setting register PWM_DUTY described later is reflected.

  The parameter PWM_EN is a parameter for setting the PWM control enable, and its initial value is “0”. Writing “0” to the parameter PWM_EN disables the PWM control, and writing “1” to the parameter PWM_EN enables the PWM control.

  FIG. 55 is a map showing the contents of the PWM clock divider register. The PWM clock frequency division register PWMCLK (index address: 41h and 42h) is a register value for setting the frequency division ratio of the PWM signal generation clock (and hence the frequency T of the PWM signal). It includes parameters PWMCLK (bit numbers 0-7 of address 41h and bit numbers 0-5 of address 42h, read / write attributes). The PWM signal generation clock is generated by frequency-dividing a reference clock having a predetermined frequency (value set by the PWM clock frequency division register + 1).

  FIG. 56 is a map showing the contents of the PWM slope control register. The PWM slope control register PWMSLOPE (index address: 43h) is a register value for setting the slope of the PWM duty (degree of change when the PWM duty is variably controlled), and is a 7-bit parameter SLOPE (bit numbers 0-6). Read / write attribute).

  FIG. 57 is a map showing the contents of the SLOPE update cycle register. The SLOPE update cycle register SLOPE_CYCLE (index address: 52h, 53h) is a register value for setting the number of slope update cycles of the PWM duty, and the 16-bit parameter SLOPE_CYCLE (bit numbers 0 to 7 of the address 52h and the address 53h bit numbers 0 to 7 and read / write attributes).

  That is, as shown in FIG. 58, the PWM duty is changed by the setting value of PWMSLOPE at every timing set by SLOPE_CYCLE × PWMCLK. When “0” is written in the PWM slope control register PWMSLOPE, there is no slope.

  FIG. 59 is a map showing the contents of the PWM duty setting register. The PWM duty setting register PWM_DUTY (index address: 44h) is a register value for setting the PWM duty (high level interval width) during external control, and is a 7-bit parameter PWM_DUTY (bit numbers 0 to 6, read / write). Attribute). By writing “0” in the parameter PWM_DUTY, the PWM signal is always low output, and by writing “111 1111b” (127d) in the parameter PWM_DUTY, the PWM signal is always high output. When the remaining register value “V” is written, one cycle is set to 127 cycles and is set to the high level only for V cycles, and is set to the low level for (127−V) cycles. Here, one cycle corresponds to one PWM signal generation clock set by the PWM frequency division clock register PWMCLK. The PWM duty setting register PWM_DUTY is only effective when the parameter PWM_HOST included in the PWM control register PWMCNT is “1” (a state in which the PWM duty is externally controlled).

  FIG. 60 is a map showing the contents of the PWMDUTY table register. The PWMDUTY table registers PWM_DTBL0 to 8 (index address: 45h to 4Dh) are tables for setting the PWM processing by the image processing IC 20-3 (a table showing a rising behavior). Bit parameters PWM_DUTY_TABLE0 to 8 (bit numbers 0 to 6, read / write attributes) are included.

  FIG. 61 is a map showing the contents of the PWM frequency coefficient register. The PWM frequency coefficient registers PWM_HCOEF0 to 3 (index address: 4Eh to 51h) are register values for setting coefficients used for weighting the luminance frequency distribution when the PWM duty is automatically controlled by the image processing IC 20-3.

  The PWM frequency coefficient register PWM_HCOEF0 includes a 3-bit parameter PWM_HCOEF0 (bit number 0 to 2, read / write attribute) and a 3-bit parameter PWM_HCOEF1 (bit number 4 to 6, read / write attribute). The PWM frequency coefficient register PWM_HCOEF1 includes a 3-bit parameter PWM_HCOEF2 (bit number 0 to 2, read / write attribute) and a 3-bit parameter PWM_HCOEF3 (bit number 4 to 6, read / write attribute). The PWM frequency coefficient register PWM_HCOEF2 includes a 3-bit parameter PWM_HCOEF4 (bit number 0 to 2, read / write attribute) and a 3-bit parameter PWM_HCOEF5 (bit number 4 to 6, read / write attribute). The PWM frequency coefficient register PWM_HCOEF3 includes a 3-bit parameter PWM_HCOEF6 (bit number 0 to 2, read / write attribute) and a 3-bit parameter PWM_HCOEF7 (bit number 4 to 6, read / write attribute).

  By writing “0” for each of the parameters PWM_HCOEF0 to 7, the weighting coefficient is set to “0 times”. Further, by writing “1”, the weighting coefficient is set to “1 time”. Similarly, by writing “2” to “7”, the weighting coefficients are “1/2 times”, “1/4 times”, “1/8 times”, “1/16 times”, “1”, respectively. / 32 times "and" 1/64 times ".

  Note that the above-described weighting process of the luminance distribution frequency is the same as that of the fifth embodiment described above, and thus a duplicate description is omitted.

  FIG. 62 is a map showing the contents of the polarity setting register. The polarity setting register POL (index address: E1h) is a register value for setting the polarity of the signal. The reserved bits RESERVE (bit numbers 2 to 4) for 3 bits and the 1-bit parameter CKPOL (bit number 5) are set. , Read / write attribute), 1-bit parameter HSPOL (bit number 6, read / write attribute), and 1-bit parameter VSPOL (bit number 7, read / write attribute).

  The parameter CKPOL is a parameter for setting the polarity of the clock signal CAMCKI with respect to the data signal CAMDI input from the camera module 10-3, and its initial value is “0”. As shown in FIG. 63, by writing “0” to the parameter CKPOL, the data signal CAMDI changes at the falling edge of the clock signal CAMCKI, and the data signal CAMDI is captured at the rising edge of the clock signal CAMCKI.

  The parameter HSPOL is a parameter for setting the polarity of the horizontal synchronization signal CAMHSI, and the initial value is “0”. By writing “0” in the parameter HSPOL, the high level section becomes the valid data section (the low level section becomes the sync section). Conversely, by writing “1” in the parameter HSPOL, the low level section becomes the valid data section (the high level section is the sync section).

  The parameter VSPOL is a parameter for setting the polarity of the vertical synchronization signal CAMVSI, and its initial value is “0”. By writing “0” in the parameter VSPOL, the high level section becomes the valid data section (the low level section becomes the sync section). Conversely, by writing “1” in the parameter VSPOL, the low level section becomes the valid data section (the high level section is the sync section).

  Note that the reserve bit RESERVE should not be changed from the initial value.

  FIG. 64 is a map showing the contents of the input interface format designation register. The input interface format designation register YUVIFSET (index address: Efh) is a register value for designating the format of the input interface. The 1-bit parameter YUVIFEN (bit number 0, read / write attribute) and the 1-bit parameter ITU656EN (Bit number 1, read / write attribute), 1-bit parameter ITU601R_I (bit number 2, read / write attribute), 1-bit parameter ITU601R_O (bit number 3, read / write attribute), and 2-bit parameter YUVORD (bit numbers 4 and 5, read / write attribute) and 2-bit parameter YUV_XST (bit numbers 6 and 7, read / write attribute) are included.

  The parameter YUVIFEN is an enable signal for the YUV interface, and its initial value is “0”. Writing “0” to the parameter YUVIFEN disables the YUV interface, and writing “1” to the parameter YUVIFEN enables the YUV interface.

  The parameter ITU656EN is a parameter for setting an input / output digital interface, and its initial value is “0”. By writing “0” in the parameter ITU656EN, the input / output digital interface becomes compatible with YCbCr with a synchronization signal, and by writing “1” in the parameter ITU656EN, the input / output digital interface supports the ITU656 format. Will be.

  The parameter ITU601R_I is a parameter for setting the range of the digital input interface, and its initial value is “0”. By writing “0” in the parameter ITU601R_I, the digital input interface has full range of Y, Cb, and Cr. On the other hand, by writing “1” in the parameter ITU601R_I, the digital input interface becomes the range of the ITU601.

  The parameter ITU601R_O is a parameter for setting the range of the digital output interface, and its initial value is “0”. By writing “0” to the parameter ITU601R_O, Y, Cb, and Cr are in the full range in the digital output interface. On the other hand, by writing “1” in the parameter ITU601R_O, the digital output interface becomes the range of ITU601.

  The parameter YUVORD is a parameter for setting the input format of Y, Cb, and Cr, and its initial value is “00b”. By writing “00b” in the parameter YUVORD, the input formats of Y, Cb, and Cr are in the order of “Y · Cb · Y · Cr”. By writing “01b” in the parameter YUVORD, the input formats of Y, Cb, and Cr are in the order of “Y · Cr · Y · Cb”. By writing “10b” in the parameter YUVORD, the input formats of Y, Cb, and Cr are in the order of “Cb · Y · Cr · Y”. By writing “11b” in the parameter YUVORD, the input formats of Y, Cb, and Cr are in the order of “Cr · Y · Cb · Y”.

  The parameter YUV_XST is a parameter for setting a delay from when the horizontal synchronization signal CAMHSI becomes valid until the data input becomes valid (see FIG. 29).

  FIG. 65 is a map showing the contents of the data format setting register. The data format setting register CAMDFM (index address: E2h) is a register value for setting the data format, and includes a 5-bit parameter DFM (bit numbers 0 to 4, read / write attributes) and a reserve bit for 3 bits. RESERVE (bit numbers 5 to 7).

  The parameter DFM is a parameter for setting the data format, and its initial value is “0”. Writing “0” to the parameter DFM sets the data format to “ITU-R BT.656-4”, and writing “1” to the parameter DFM sets the data format to “YCbCr with sync signal”. Is done. Note that writing values other than those described above to the parameter DFM is prohibited.

  In the above embodiment, the case where the present invention is applied to an image processing apparatus that performs luminance conversion processing on each frame image obtained by moving image capturing has been described as an example. The target is not limited to this, and the input image is subjected to luminance conversion processing, such as an image processing device that performs luminance conversion processing on a received image of a television broadcast, a playback image of a DVD, or a still image such as a photograph. The present invention can be applied to an image processing circuit that generates the output image, a semiconductor device formed by integrating the output processing circuit, and an image processing apparatus using the image processing circuit.

  The configuration of the present invention can be variously modified within the scope of the present invention in addition to the above embodiment.

  The present invention relates to an image processing circuit that performs luminance conversion processing on an input image to generate a desired output image, a semiconductor device integrated with the image processing circuit, and an image processing device using the image processing circuit. This is a useful technique for improving the performance.

These are block diagrams which show 1st Embodiment of the image processing apparatus which concerns on this invention. FIG. 4 is a diagram for explaining the overall flow of image processing. These are the flowcharts for demonstrating the calculation process of a luminance conversion coefficient. These are the figures for demonstrating the aspect of area division. These are the figures for demonstrating the state by which the virtual area was set. These are figures for demonstrating the aspect of a low-pass filter. These are the figures for demonstrating bilinear calculation. These are block diagrams which show 2nd Embodiment of the image processing apparatus which concerns on this invention. These are block diagrams which show 3rd Embodiment of the image processing apparatus which concerns on this invention. These are schematic diagrams showing the correlation between the luminance correction amount and the intensity of the noise removal processing. These are block diagrams which show 4th Embodiment of the image processing apparatus which concerns on this invention. These are schematic diagrams showing the correlation between the luminance correction amount and the strength of the edge enhancement processing. These are block diagrams which show 5th Embodiment of the image processing apparatus which concerns on this invention. These are figures which show one structural example of the backlight drive device which concerns on this invention. These are figures which show an example of the brightness | luminance histogram of an input image. These are figures which show the conversion table used by the duty determination part 204. FIG. These are the figures for demonstrating the PWM drive of a backlight. These are block diagrams which show 6th Embodiment of the image processing apparatus which concerns on this invention. FIG. 3 is a block diagram illustrating a configuration example of an image processing unit 21. FIG. 4 is a block diagram illustrating a configuration example of a correction coefficient calculation unit 214. FIG. 10 is a block diagram showing another configuration example of the correction coefficient calculation unit 214. These are block diagrams which show 7th Embodiment of the image processing apparatus which concerns on this invention. These are block diagrams which show the example of 1 structure of image processing IC20-3. These are terminal function tables of the image processing IC 20-3. These are terminal function tables (continued) of the image processing IC 20-3. These are timing charts showing "ITU-R BT.656" input format (horizontal bottom). These are timing charts showing the "ITU-R BT.656" input format (vertical direction) in NTSC. These are timing charts showing the “ITU-R BT.656” input format (vertical bottom) in PAL. These are timing charts showing "YCbCr with synchronization signal" 8-bit input format (horizontal direction). These are timing charts showing "YCbCr with synchronization signal" 8-bit input format (vertical direction). FIG. 4 is a diagram showing an I2C interface format. These are maps showing the contents of the initialization register SRST. Is a map showing the contents of the parameter update register PARAMSET. Is a map showing the contents of the mode register MODE. These are logical tables showing the relationship between the parameter MODE and the operation mode. These are maps showing the contents of the camera X direction pixel size setting register DXSIZE. These are maps showing the contents of the camera Y direction pixel size setting register DYSIZE. These are maps showing the contents of the image correction effective area X-direction start position setting register AIEXST. These are maps showing the contents of the image correction effective area Y-direction start position setting register AIEYST. These are maps showing the contents of the image correction effective area X-direction size setting register AIEXSIZE. These are maps showing the contents of the image correction effective area Y-direction size setting register AIEYSIZE. FIG. 5 is a diagram schematically showing an image correction effective area of input image data. These are maps showing the brightness correction strength setting register STRENGTH. These are maps showing the contents of the color difference correction intensity setting register UV_STRENGTH. These are maps showing the noise suppression setting register NOISE_SUP. These are maps showing the contents of the edge enhancement filter setting register EDG_CNT. Is a map showing the contents of the response time setting register RESP_SET. These are maps showing the contents of the post filter enable register PFLT_EN. These are maps showing the contents of the output gamma characteristic registers POFLT0 to POFLT8. FIG. 4 is a diagram showing an output gamma characteristic. These are maps showing the contents of the luminance distribution setting register LFREQSET. These are maps showing the contents of the luminance distribution registers LFREQ0-7. Is a map showing a luminance distribution parameter register LFREQ_TOTAL. These are maps showing the contents of the PWM control register. These are maps showing the contents of the PWM clock divider register. These are maps showing the contents of the PWM slope control register. Is a map showing the contents of the SLOPE update cycle register. These are figures which show the mode of the variable control of PWM duty. These are maps showing the contents of the PWM duty setting register. These are maps showing the contents of the PWMDUTY table register. These are maps showing the contents of the PWM frequency coefficient register. FIG. 4 is a map showing the contents of a polarity setting register. These are the figures for demonstrating the polarity setting by parameter CKPOL. FIG. 4 is a map showing the contents of an input interface format designation register. FIG. 4 is a map showing the contents of a data format setting register.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 1-2, 1-3, 1-4, 1-5 Image processing apparatus 10 Imaging part (imaging means)
10-2 Image Generation Device 10-3 Camera Module 11 Luminance Conversion Processing Unit 12 Luminance Conversion Coefficient Calculation Unit 13 Luminance Conversion Coefficient Storage Unit 14 Output Unit 14-2 Image Output Device (Liquid Crystal Display)
15, 15-2 Noise filter processing unit 16 Noise filter control unit 17, 17-2 Filter coefficient setting unit 18 Edge enhancement processing unit 19 Luminance detection unit 20, 20-2, 20-3 Image processing IC (semiconductor device)
21 image processing unit 21a color correction unit 211 luminance correction unit 212 color correction unit 213 luminance histogram acquisition unit 214 correction coefficient calculation unit 214a first correction table 214b second correction table 214c reference correction table 214d adjustment coefficient calculation unit 22 color correction control unit 30, 30-2 Control unit (CPU)
40 Illuminance Sensor 100 Luminance Detection Unit 200 Drive Unit 201 First Adder 202 Multiplier 203 Second Adder 204 Duty Determination Unit 300 Liquid Crystal Display A1 Color Correction Unit A2 Brightness Discrimination Unit A3 Image Enhancement Unit A4 Register A5 I2C Interface Unit A6 Edge Emphasis / gamma correction part A7 PWM signal generation control part A8 Timing generation part

Claims (5)

A correction effective area setting unit for setting a correction effective area of the input image;
A luminance conversion coefficient calculation unit that calculates a luminance conversion coefficient for each area after dividing only the correction effective area of the input image into a plurality of areas;
A luminance conversion processing unit that performs luminance conversion processing according to the luminance conversion coefficient for each area on the pixels constituting each area;
An image processing circuit comprising:   The image processing circuit according to claim 1, wherein the correction effective area setting unit includes a register that stores a setting value for determining the correction effective area.   3. The image processing circuit according to claim 2, wherein the set value includes a register value indicating a start position of the correction effective area and a register value indicating a pixel size of the correction effective area.   A semiconductor device in which the image processing circuit according to claim 1 is integrated.   An image processing apparatus comprising: the semiconductor device according to claim 4; and an image input source that supplies the input image to the semiconductor device.

Images