JP2020061647A - Image processing apparatus, display unit, and image processing method - Google Patents

Abstract

To provide a technology that can prevent the hue and chroma of an image indicated by image data from changing before and after processing to correct brightness is performed on the image data.SOLUTION: An image processing apparatus comprises: a generation unit that generates, based on image data represented by a red component, a green component, and a blue component, a first brightness value for the brightness specified in the image data; a correction unit that corrects the first brightness value to generate a second brightness value; a determination unit that determines a gain for the image data, based on a first ratio between the first brightness value and the second brightness value, and a second ratio between the maximum value of the red component, the green component, and the blue component and an upper limit value that can be taken by the maximum value; and an amplification unit that amplifies the image data with the gain determined by the determination unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing device, a display device, and an image processing method.

Patent Document 1 describes a technique of emphasizing a contour in an image represented by image data represented by RGB components. Here, R means red, G means green, and B means blue.
In this technique, first, the representation format of image data represented by each component of RGB is converted into the representation format represented by each component of hue, saturation, and lightness. Subsequently, only the lightness of the hue, the saturation, and the lightness is subjected to the edge enhancement processing to generate the corrected image data. Then, the expression form of the corrected image data is changed from the expression form shown by each component of hue, saturation, and lightness to the expression form shown by each component of RGB.

JP-A-5-298442

  In the technique described in Patent Document 1, when the expression form of the corrected image data is changed to the expression form represented by each component of RGB, at least one of the RGB components is saturated depending on the correction amount of lightness, The balance of the RGB components may be lost, and the hue or saturation may change significantly before and after the correction. Therefore, there is a demand for a technique capable of suppressing a large change in the hue and saturation of the image indicated by the image data before and after the process of correcting the brightness such as the brightness is performed on the image data.

  One aspect of the image processing apparatus according to the present invention generates a first brightness value for the brightness specified by the image data, based on the image data represented by the red component, the green component, and the blue component. A generation unit, a correction unit that corrects the first brightness value to generate a second brightness value, a first ratio of the first brightness value and the second brightness value, and the red component. A determination unit that determines a gain for the image data based on a relationship between a maximum value of the green component and the blue component and an upper limit value that can be taken by the maximum component; An amplification unit that amplifies the image data with the gain determined by the unit.

  One aspect of the image processing method according to the present invention generates a first brightness value for the brightness specified by the image data, based on image data represented by a red component, a green component, and a blue component. Correcting the first brightness value to generate a second brightness value, a first ratio of the first brightness value to the second brightness value, the red component, the green component, and the blue color. The gain for the image data is determined based on the relationship between the maximum value of the components and the second ratio of the upper limit value that the maximum value component can have, and the gain for the image data is determined, and the image data is amplified by the gain.

It is a figure which shows an example of the image processing apparatus 100 which concerns on 1st Embodiment. 6 is a flowchart for explaining the operation of the image processing apparatus 100. 7 is a flowchart for explaining an example of a gain determination operation. It is a figure which shows an example of RGB image data. It is a figure which shows an example of 1st brightness value Y1. It is a figure which shows an example of 2nd luminance value Y2. It is a figure which shows the example of saturation of RGB image data. It is a figure which shows the modification 1. FIG. 1 is a diagram showing an example of a projector 1 having an image processing device 100. 6 is a diagram showing an example of a projection unit 700. FIG.

<A: First Embodiment>
<A1: Overview of image processing apparatus 100>
FIG. 1 is a diagram showing an example of an image processing apparatus 100 according to the first embodiment. The image processing apparatus 100 includes a color system conversion unit 101, a sharpness processing unit 102, and a gain adjustment unit 103. In the image processing apparatus 100, the image data represented by the RGB color system is distributed to the color system conversion unit 101 and the gain adjustment unit 103. Hereinafter, the image data represented by the RGB color system will be referred to as “RGB image data”. RGB image data is represented by a red component, a green component, and a blue component.

  The color system conversion unit 101 is an example of a generation unit. The color system conversion unit 101 generates a brightness value specified by the RGB image data based on the RGB image data. The color system conversion unit 101 generates a Y component used in the YUV color system as a luminance value. In the YUV color system, Y means luminance, U means color difference between B component and Y component, and V means color difference between R component and Y component. The Y component means a luminance component.

  The sharpness processing unit 102 is an example of a correction unit. The sharpness processing unit 102 performs sharpness processing on the Y component generated by the color system conversion unit 101. In the sharpness processing, the Y component is corrected so that a portion having a large change in the Y component, for example, a portion showing a contour is emphasized. The sharpness process is executed in pixel units. The sharpness processing is an example of image quality correction processing for correcting the image quality of the image represented by the RGB image data.

  The gain adjusting unit 103 adjusts the gain for the RGB image data distributed to the gain adjusting unit 103. The gain adjusting unit 103 includes a determining unit 104 and an amplifying unit 105. The determination unit 104 and the amplification unit 105 receive the RGB image data distributed to the gain adjustment unit 103.

  The determining unit 104 determines the gain for the RGB image data. The amplification unit 105 amplifies the RGB image data with a gain determined by the determination unit 104.

<A2: Overall Operation of Image Processing Device 100>
FIG. 2 is a flowchart for explaining the operation of the image processing apparatus 100.
Upon receiving the RGB image data, the color system conversion unit 101 generates a Y component from the RGB image data in step S11.

For example, the color system conversion unit 101 generates a Y component using a conversion formula that converts RGB image data into a Y component. An example of the conversion formula is shown below.
Y = 0.2990 * R + 0.5870 * G + 0.1140 * B.
The above-mentioned conversion formula is a part of a known conversion formula used when converting RGB image data into image data represented by the YUV color system.

Subsequently, in step S12, the sharpness processing unit 102 corrects the value of the Y component by performing the sharpness processing on the Y component. As described above, since the sharpness processing is performed only on the Y component, the processing amount of the sharpness processing can be reduced and the configuration of the sharpness processing unit 102 can be simplified as compared with the configuration in which the sharpness correction is performed for each of the RGB components, for example. .
Hereinafter, the value of the Y component before correction is referred to as “first luminance value Y1”, and the value of the Y component after correction is referred to as “second luminance value Y2”. The first brightness value Y1 is an example of a first brightness value for the brightness specified by the RGB image data. The second brightness value Y2 is an example of the second brightness value.

  Subsequently, in step S13, the determination unit 104 determines the gain for the RGB image data based on the first luminance value Y1, the second luminance value Y2, and the RGB image data. An example of the gain determining operation in step S13 will be described later.

  Subsequently, in step S14, the amplification unit 105 amplifies the RGB image data with the gain determined by the determination unit 104.

<A3: Gain determination operation>
FIG. 3 is a flowchart for explaining an example of the gain determination operation in step S13. Here, in order to simplify the description, the R component, the G component, and the B component in the RGB image data are in the state shown in FIG. 4, the first luminance value Y1 is in the state shown in FIG. 5, and the second luminance value is It is assumed that Y2 is in the state shown in FIG. The R component, the G component, and the B component in the RGB image data are not limited to the states shown in FIG. 4, the first luminance value Y1 is not limited to the state shown in FIG. 5, and the second luminance value Y2 is the state shown in FIG. Not limited to

  When each component of RGB is represented by 8 bits in RGB image data, the R component, the G component, and the B component are expressed in the range of 0 to 255, respectively. In this case, the upper limit value MAXL and the lower limit value MINL of the R component, the G component, and the B component are 255 and 0, respectively. The number of bits of each component of RGB is not limited to 8 bits, the upper limit value MAXL is not limited to 255, and the lower limit value MIN is not limited to 0.

  In step S21, the determination unit 104 calculates the ratio of the first brightness value Y1 shown in FIG. 5 and the second brightness value Y2 shown in FIG. 6 as the first ratio. In the present embodiment, the determination unit 104 sets the value of the ratio of the second brightness value Y2 to the first brightness value Y1, specifically, the second brightness value Y2 / first brightness value Y1 as the value of the first ratio. calculate.

Subsequently, in step S22, the determination unit 104 determines the ratio of the maximum value MAX of the R component, the G component, and the B component in the RGB image data to the upper limit value MAXL that the component of the maximum value MAX can take. Calculate as a ratio. In the present embodiment, the determination unit 104 calculates the value of the ratio of the upper limit value MAXL to the maximum value MAX, that is, the upper limit value / maximum value, as the value of the second ratio.
In the example shown in FIG. 4, the determination unit 104 determines the value of the G component as the maximum value, and determines the upper limit value MAXL of the G component that takes the maximum value MAX as the upper limit value. The upper limit value MAXL of each component of RGB is a fixed value and is held in the determining unit 104 in advance. Then, the determination unit 104 calculates the upper limit value of the G component / the value of the G component as the value of the second ratio.
The execution order of step S21 and step S22 may be reversed.

  Subsequently, in step S23, the determination unit 104 determines the gain for the RGB image data based on the relationship between the first ratio and the second ratio. In the present embodiment, the determination unit 104 determines the smaller value of the first ratio value and the second ratio value as the gain for the RGB image data.

  The value of the second ratio is the maximum value of the gain in which only the brightness can be changed without changing the ratio of the RGB components in the RGB image data. When the gain for the RGB image data exceeds the value of the second ratio, as illustrated in FIG. 7, at least one of the RGB components is saturated without exceeding the upper limit value, and the RGB components of the RGB components are saturated. The ratio is changed, and the hue and saturation of the image represented by the RGB image data are greatly changed.

According to the image processing apparatus 100 and the image processing method according to the present embodiment, the gain for the RGB image data is determined based on the relationship between the first ratio and the second ratio.
Therefore, if the value of the first ratio, which is the gain determined by the sharpness process, is smaller than the value of the second ratio, the determination unit 104 determines the value of the first ratio as the gain for the RGB image data, and the sharpness is determined. While making use of the processing result, it is possible to suppress a large change in hue and saturation of the image represented by the RGB image data.
Further, when the value of the first ratio is larger than the value of the second ratio, if the determination unit 104 determines the value of the second ratio as the gain for the RGB image data, the sharpness is increased by increasing the brightness of the RGB image data. While realizing, it is possible to suppress the hue and saturation of the image represented by the RGB image data from largely changing.

  When the value of the first ratio matches the value of the second ratio, the determination unit 104 may determine the value of the first ratio as the gain for the RGB image data, or the value of the second ratio. You may.

  When the value of each of the RGB components of the RGB image data is 0, the determining unit 104 does not determine the gain for the RGB image data, and causes the amplifying unit 105 to determine the value of each of the RGB components to the second luminance. The RGB image data that is equal to the value Y2 is output. Also in this case, the RGB image data input to the image processing apparatus 100 and the RGB image data output from the image processing apparatus 100 maintain the ratio of the RGB components, and the hue and saturation of the image significantly change. Can be suppressed.

<B: Modification>
In the first embodiment, for example, the configuration exemplified below may be adopted.

<B1: Modification 1>
In the first embodiment, contrast correction processing may be used instead of sharpness processing. FIG. 8 is a diagram illustrating an example of the image processing apparatus 100 in which the contrast correction unit 102a is used instead of the sharpness processing unit 102. In FIG. 8, the same components as those shown in FIG. 1 are designated by the same reference numerals.

The contrast correction unit 102a is another example of the correction unit. The contrast correction unit 102a performs contrast correction processing on the Y component generated by the color system conversion unit 101. In the contrast correction process, the difference between light and dark in the image represented by the image data is corrected. The contrast correction process is executed in pixel units. The contrast correction process is another example of the image quality correction process.
Also in the first modification, it is possible to increase the brightness of the RGB image data and suppress a large change in the hue and saturation of the image represented by the RGB image data.
The image quality correction process is not limited to the sharpness process and the contrast correction process, and may be any process that corrects the brightness value specified by the image data.

<B2: Modification 2>
In the first embodiment and the first modification, as the value of the first ratio, instead of the value of the ratio of the second brightness value Y2 to the first brightness value Y1, the ratio of the first brightness value Y1 to the second brightness value Y2 is set. A value is used, and the value of the ratio of the maximum value MAX to the upper limit MAXL may be used as the value of the second ratio, instead of the value of the ratio of the upper limit MAXL to the maximum value MAX. In this case, the determining unit 104 determines the reciprocal of the larger value of the first ratio value and the second ratio value as the gain for the RGB image data.

<B3: Modification 3>
In the first embodiment and the modified examples 1 and 2, the color system conversion unit 101 generates the lightness based on the RGB image data instead of generating the first luminance value Y1 based on the RGB image data. Good. For example, the color system conversion unit 101 generates, as the lightness, the V component used in the HSV color system. In the HSV color system, H means hue, S means saturation, and V means lightness. Therefore, the V component means the lightness component.
In this case, the sharpness processing unit 102, the contrast correction unit 102a, and the determination unit 104 use the V component instead of the Y component.
The value of the V component before correction by the sharpness processing unit 102 or the contrast correction unit 102a is another example of the first brightness value. The value of the V component after being corrected by the sharpness processing unit 102 or the contrast correction unit 102a is another example of the second brightness value.
The first brightness value and the second brightness value may each be a value of a component for brightness in another color system, for example, a lightness L component in the HLS color system.

<B4: Modification 4>
The image processing apparatus 100 shown in the first embodiment and the modified examples 1 to 3 may be mounted on a display device such as a projector.

  FIG. 9 is a diagram illustrating an example of the projector 1 including the image processing device 100. The projector 1 includes an image processing device 100, a bus 200, a storage unit 300, a control unit 400, an operation unit 500, a reception unit 600, and a projection unit 700.

  The image processing apparatus 100, the storage unit 300, the control unit 400, the operation unit 500, the receiving unit 600, and the projection unit 700 are connected to each other via a bus 200. The storage unit 300 is a computer-readable recording medium. The storage unit 300 stores a program that defines the operation of the projector 1 and various types of information. The control unit 400 is a computer such as a CPU (Central Processing Unit). The control unit 400 may include one or a plurality of processing devices. The control unit 400 controls the projector 1 by reading and executing the program stored in the storage unit 300.

  The operation unit 500 is, for example, various operation buttons, operation keys, or a touch panel. The operation unit 500 receives a user's input operation. The receiving unit 600 receives the RGB image data. The receiving unit 600 outputs the RGB image data to the image processing device 100.

  The image processing apparatus 100 performs image quality correction processing, such as sharpness processing or contrast correction processing, on the RGB image data. The image quality correction process in the image processing apparatus 100 is controlled by the control unit 400, for example, according to an input operation received by the operation unit 500. The projection unit 700 projects an image on a projection surface (not shown) according to the RGB image data output by the image processing apparatus 100.

  FIG. 10 is a diagram showing an example of the projection unit 700. The projection unit 700 includes a light valve drive unit 11, a light source drive unit 12, a light source 13, a red liquid crystal light valve 14R, a green liquid crystal light valve 14G, a blue liquid crystal light valve 14B, and a projection optical system 15. And, including. Hereinafter, when it is not necessary to distinguish the red liquid crystal light valve 14R, the green liquid crystal light valve 14G, and the blue liquid crystal light valve 14B from each other, a red liquid crystal light valve 14R and a green liquid crystal light valve 14G are used. Each of the blue liquid crystal light valves 14B is simply referred to as "liquid crystal light valve 14".

  The light valve drive unit 11 drives the liquid crystal light valve 14 based on the RGB image data. The light source drive unit 12 drives the light source 13.

  The light source 13 is a xenon lamp, an ultrahigh pressure mercury lamp, an LED (Light Emitting Diode), a laser light source, or the like. The light emitted from the light source 13 is reduced in variation in the luminance distribution by an integrator optical system (not shown), and then separated into red, green and blue light components which are the three primary colors of light by a color separation optical system (not shown). To be done. The red color light component enters the red liquid crystal light valve 14R. The green colored light component enters the green liquid crystal light valve 14G. The blue colored light component enters the blue liquid crystal light valve 14B.

  The liquid crystal light valve 14 is composed of a liquid crystal panel or the like in which liquid crystal exists between a pair of transparent substrates. The liquid crystal light valve 14 has a rectangular pixel region 14a including a plurality of pixels 14p arranged in a matrix. The liquid crystal light valve 14 can apply a drive voltage to the liquid crystal for each pixel 14p. When the light valve drive unit 11 applies a drive voltage based on the RGB image data input from the image processing device 100 to each pixel 14p, each pixel 14p is set to the light transmittance based on the drive voltage. Therefore, the light emitted from the light source 13 is modulated by passing through the pixel region 14a, and an image based on the RGB image data is formed for each color light.

  The images of the respective colors are combined for each pixel 14p by a color combining optical system (not shown) to generate a projection image which is a color image. The projection image is projected on the projection surface by the projection optical system 15.

Although the liquid crystal light valve 14 is used as an example of the light modulation device, the light modulation device is not limited to the liquid crystal light valve and can be appropriately changed. For example, the light modulation device may have a configuration using three reflective liquid crystal panels. Further, the light modulation device may have a configuration such as a system using one liquid crystal panel, a system using three digital mirror devices (DMD), a system using one digital mirror device, or the like. When only one liquid crystal panel or DMD is used as the light modulation device, members corresponding to the color separation optical system and the color synthesis optical system are unnecessary. In addition to the liquid crystal panel and the DMD, a configuration capable of modulating the light emitted by the light source 13 can be adopted as the light modulator.
The display device is not limited to the projector 1 and may be a direct-view display such as a liquid crystal display.

<B5: Modification 5>
In the first embodiment and Modifications 1 to 4, all or some of the elements included in the image processing apparatus 100 are hardware by an electronic circuit such as an FPGA (field programmable gate array) or an ASIC (Application Specific IC). Or may be realized by a computer such as a CPU reading and executing a program stored in a storage device.

  DESCRIPTION OF SYMBOLS 1 ... Projector, 11 ... Light valve drive part, 12 ... Light source drive part, 13 ... Light source, 14 ... Liquid crystal light valve, 15 ... Projection optical system, 100 ... Image processing device, 101 ... Color system conversion part, 102 ... Sharpness Processing unit, 102a ... Contrast correction unit, 103 ... Gain adjustment unit, 104 ... Determination unit, 105 ... Amplification unit, 200 ... Bus, 300 ... Storage unit, 400 ... Control unit, 500 ... Operation unit, 600 ... Receiving unit, 700 … Projection unit.

Claims (8)

A generation unit that generates a first brightness value for the brightness specified by the image data based on the image data represented by the red component, the green component, and the blue component;
A correction unit that corrects the first brightness value to generate a second brightness value;
A first ratio of the first brightness value and the second brightness value, a maximum value of the red component, the green component, and the blue component, and an upper limit value that can be taken by the component of the maximum value. A determining unit that determines a gain for the image data based on the relationship between
An amplification unit that amplifies the image data with the gain determined by the determination unit,
An image processing apparatus including. The value of the first ratio is a value of a ratio of the second brightness value to the first brightness value,
The value of the second ratio is a value of the ratio of the upper limit value to the maximum value,
The determination unit determines the smaller one of the value of the first ratio and the value of the second ratio as the gain,
The image processing apparatus according to claim 1. The value of the first ratio is a value of a ratio of the first brightness value to the second brightness value,
The value of the second ratio is a value of the ratio of the maximum value to the upper limit value,
The determining unit determines the larger reciprocal of the value of the first ratio and the value of the second ratio as the gain.
The image processing apparatus according to claim 1. The first brightness value and the second brightness value are values of a luminance component in the YUV color system, respectively.
The image processing apparatus according to claim 1. The first brightness value and the second brightness value are values of the lightness component in the HSV color system, respectively.
The image processing apparatus according to claim 1. The correction unit corrects the first brightness value in an image quality correction process for correcting the image quality of the image represented by the image data, and generates the second brightness value.
The image processing apparatus according to claim 1.   A display device including the image processing device according to claim 1. Generating a first brightness value for the brightness specified by the image data, based on the image data represented by the red component, the green component, and the blue component,
Correcting the first brightness value to generate a second brightness value,
A first ratio of the first brightness value and the second brightness value, a maximum value of the red component, the green component, and the blue component, and an upper limit value that can be taken by the component of the maximum value. The gain for the image data is determined based on the relationship between
Amplifying the image data with the gain,
Image processing method.

Images