JP2015099980A - Image processing system, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate image data of image quality seemingly having been viewed actually at a dark place.SOLUTION: A luminance estimation unit 1116 calculates a luminance in imaging of image data displayed within a screen. When the luminance in imaging calculated by the luminance estimation unit 1116 is not larger than a threshold, a color temperature change unit 1119 and a color density change unit 1120 execute correction processing to raise the color temperature of the image data and correction processing to reduce the density of the image data on the basis of the luminance in imaging.

Description

  The present invention relates to a technique for correcting image data shot in a dark place.

  Conventionally, when viewing a display device in a bright environment, the image data is processed so that the viewer can easily see it by increasing the brightness, increasing the color saturation, or increasing the contrast depending on the illuminance around the display device. To be done. Furthermore, the color temperature of the display device is changed in accordance with the sunlight around the display device or the color temperature of illumination. This is because, due to eye characteristics, there is a phenomenon in which the color temperature of the display device looks high in an environment where the color temperature is low, and the color temperature of the display device appears low in an environment where the color temperature is high. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for measuring the ambient color temperature with a sensor attached to the display device and adjusting the color temperature of the display device in the same direction as the ambient color temperature using an LUT.

  By the way, as another characteristic of the eye, it is known that the working ratio of the two photoreceptor cells of the cone cell and the rod cell changes according to the brightness. In bright places, pyramidal cells mainly work, and in dark places, rod cells mainly work. Such changes in eye characteristics due to brightness are referred to as Purkinje transition. The cone cells can recognize three colors of RGB depending on the difference between the cells, whereas the rod cells can recognize only monochrome, and the frequency at which the center of sensitivity is different. Therefore, when the proportion of the body vision increases in a dark place, the color density and the color temperature appear to be different due to Purkinje transition.

  Therefore, Patent Documents 2, 3 and 4 disclose techniques for correcting color temperature and color density with respect to Purkinje transition that occurs according to the viewing environment. That is, when the environmental luminance and the APL value are lower than a certain level, image processing is performed so that the color temperature is increased and the color density is increased. Here, APL is an abbreviation of Average Picture Level and is a value obtained by averaging the number of gradations of image data of one frame.

  Further, Non-Patent Document 1 discloses that tone mapping is performed on the color of image data in accordance with the transmitted light intensity of red due to the influence of dark illuminance on the actual image data. Patent Document 5 discloses a technique for performing image correction with the lightness having an inverse S-characteristic and the chromaticity having an S-characteristic in order to match the nonlinearity of the photographing apparatus and the nonlinearity of the eye.

JP 2001-337665 A JP 2006-285063 A JP 2007-248936 A JP 2007-248935 A JP 2007-158388 A

The 243rd meeting of the Institute of Image Electronics Engineers of Japan "Tone Reproduction Method Considering Human Visual Characteristics in Mesopic Vision"

  However, when image data captured in a dark environment is viewed in a bright environment, the Purkinje transition does not occur in the image capturing apparatus unlike a human eye, and no Purkinje transition occurs because the display is viewed in a bright environment. In this case, there is a problem in that the display on the display device is viewed differently from the way it is viewed in a dark environment.

  In the technique disclosed in Patent Document 1, since the control is performed only by the color temperature of the display environment, it is not possible to cope with the above-described problem with respect to the Purkinje transition. Further, in the techniques disclosed in Patent Documents 2, 3, and 4, the influence of Purkinje transition is corrected only when the display and the surroundings are dark, so it is not possible to deal with when the surroundings are bright.

  In the technique disclosed in Non-Patent Document 1, since the color of the entire image data is changed according to the illuminance at the shooting location, not only the dark part of the image data but also the color of the bright part of the image data becomes dull. Further, since the influence of environmental illuminance at the time of viewing is not taken into consideration, when the processed image data is viewed in a dark environment, the correction of Purkinje transition is applied twice, and red becomes a completely strange color.

  Further, in the technique disclosed in Patent Document 5, image data is corrected without considering the influence of ambient light at the time of shooting and viewing, so that correction is applied in the opposite direction depending on the state of ambient light. May end up.

  Accordingly, an object of the present invention is to generate image data having an image quality as if it was actually viewed in a dark place.

  The image processing apparatus according to the present invention includes a photographing luminance calculation unit that calculates luminance at the time of photographing of image data displayed in a screen, and the photographing luminance calculated by the photographing luminance calculation unit is a first luminance. If it is equal to or lower than the threshold value, at least one of correction processing for increasing the color temperature of the image data and correction processing for decreasing the color density of the image data is executed based on the luminance at the time of shooting. And a correction means for performing the above-described correction.

  According to the present invention, it is possible to generate image data having an image quality as if it was actually viewed in a dark place.

It is a figure which shows CIE standard spectral luminous efficiency. It is a figure which shows the spectral sensitivity of a photoreceptor cell. It is a figure for demonstrating the relationship between the real image data in the real environment of different brightness, and the display image data in different ambient luminance. It is a figure which shows the brightness | luminance detection range of the cone view and the rod view with respect to the average brightness | luminance in a visual field. It is a figure which shows the ratio of the rod cell which influences the sensitivity of the synthetic vision of the cone cell and the rod cell with respect to the average brightness | luminance in a visual field. It is a figure which shows the ratio which feels the color intensity with respect to the assumption average brightness | luminance at the time of imaging | photography. It is a figure which shows the ratio which feels the color density with respect to the average luminance in a visual field at the time of viewing. It is a figure which shows the shift | offset | difference of color temperature with respect to the average luminance in a visual field. It is a figure which shows the result of having calculated the average brightness | luminance in a visual field from screen edge illumination intensity and an average gradation. It is a figure which shows the structure of the display apparatus which concerns on the 1st Embodiment of this invention. 1 is a diagram illustrating a detailed configuration of a color correction circuit according to a first embodiment of the present invention. It is a figure which shows the ratio of the rod cell which influences the sensitivity of the synthetic vision of the cone cell and the rod cell with respect to the average brightness | luminance in a visual field. It is a figure which shows the detailed structure of the color correction circuit in the 3rd Embodiment of this invention. It is a figure which shows the detailed structure of the color correction circuit in the 4th Embodiment of this invention. It is a figure which shows the structure of the image quality adjustment apparatus which concerns on the 5th Embodiment of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram showing CIE standard spectral luminous efficiency. In FIG. 1, the horizontal axis represents the wavelength of light, and the vertical axis represents the spectral luminous efficiency. In FIG. 1, 11 is a visibility curve for scotopic vision, and 12 is a visibility curve for photopic vision. As shown at 12 in FIG. 1, the peak wavelength of sensitivity is 570 nm in photopic vision, and as shown in 11 in FIG. 1, the peak wavelength of sensitivity is 500 nm in dark vision. It can be seen that in dark places, the ratio of rod vision increases, so the short wavelength side is felt brighter and the color of the object appears bluish.

  FIG. 2 is a diagram showing the spectral sensitivity of photoreceptor cells. The source of FIG. 2 is Chapter 1 of Volume 2 of the Knowledge Base of the Institute of Electronics, Information and Communication Engineers. In FIG. 2, 21 is a spectral sensitivity curve of rod cells, 22 is a spectral sensitivity curve of blue cone cells, 23 is a spectral sensitivity curve of green cone cells, and 24 is a spectrum of red cone cells. It is a sensitivity curve.

  As shown in FIG. 2, it can be seen that the pyramidal cells have the sensitivity of the three primary colors of blue, green and red, whereas the rod cells have the sensitivity of one color. It can be seen that the color component appears to decrease because the proportion of rod vision increases in dark places. In other words, it can be seen that the color becomes lighter as it gets darker.

  FIG. 3 is a diagram for explaining the relationship between actual image data in an actual environment with different brightness and display image data at different ambient luminance. In FIG. 3, 31 is real image data in a bright real environment, 33 is an environment in which the periphery of the display device is dark, and 32 is bright display image data. Reference numeral 34 denotes real image data in a dark real environment, reference numeral 35 denotes an environment in which the periphery of the display device is bright, and reference numeral 36 denotes dark display image data. When the real image data 31 photographed in a bright real environment is viewed as the display image data 32 in the dark environment 33, Purkinje transition occurs because the environment is dark, and the display image data 32 looks blue and the color appears light.

  On the other hand, in the actual image data 34 actually photographed in a dark actual environment, the actually viewed landscape looks at the state where the Purkinje transition has occurred. However, since no Purkinje transition occurs in the camera and no Purkinje transition occurs in the bright environment 35, as a result, the display image data 36 also appears as image data in which no Purkinje transition has occurred. In this case, the appearance changes depending on whether the dark place is actually viewed or the display device.

  Therefore, in the present embodiment, when the actual image data 34 is image data within a range where Purkinje transition occurs and the bright environment 35 has brightness within a range where Purkinje transition does not occur, the color temperature of the display image data 36 To increase the color density. As a result, a display state is created as if the user is looking in a dark environment while viewing in a bright environment 35.

  FIG. 4 is a diagram illustrating the luminance detection range of the cone view and the rod view with respect to the average luminance in the visual field. In FIG. 4, the vertical axis represents the average luminance within the visual field. The visual luminance detection range, that is, the dynamic range of the eye is said to be 10 6 to the 10th power. On the other hand, the luminance range within the visual field when watching TV in the living room is between several candela and several hundred candela. The luminance range in the field of view when looking at the projector in the theater room is 0. It is between several candela and several tens of candela.

  The luminance detection range for cone vision is 0.1 candela or more, and the luminance detection range for rod vision is 10 candela or less. In the region from 0.1 to 10 candela, both photoreceptors work without saturation.

Instead of switching from a cone view to a rod view like a switch, it is seen with a synthetic sensitivity consisting of a composite view of cone cells and rod cells. When the eye adaptation is completed, when the average luminance in the visual field is about 1 Cd / m ² , the ratio of the cone-view to the rod-view on the sensitivity of the composite view is about 1: 1, approximately half each. It is said that.

  In the theater room, viewing is performed almost within the luminance detection range of synthetic vision, so viewing is performed with color characteristics obtained by combining the color characteristics of both photoreceptor cells. In the living room, viewing is mostly performed in the pyramid luminance detection range, and only in the dark living room, viewing is performed in the synthetic vision luminance detection range.

On the other hand, when looking directly at the actual object, if it is less than 10 candela, it is the luminance detection range of the body vision, so the color characteristics felt at that time correspond to the color characteristics under the influence of the body vision It is. When this is viewed in an environment of 10 Cd / m ² or more, since it is viewed with color characteristics almost only in a pyramid view, the color of the dark portion of the image data is particularly reddish and the color is felt dark.

Therefore, in a living environment, when viewing a display device, in an image area in which the displayed image data can be inferred to be 10 Cd / m ² or less (first threshold) when viewing the real object. The color temperature is corrected to be high, or the process of reducing the color density is performed. In addition, you may determine whether it is a living environment according to whether the average brightness | luminance in a visual field is 10 Cd / m < ² > or more.

FIG. 5 is a diagram showing the ratio of rod cells that influences the sensitivity of the synthetic vision of cone cells and rod cells with respect to the average luminance within the visual field. In FIG. 5, the horizontal axis represents the average luminance within the visual field, and the vertical axis represents the ratio of rod cells that affects the sensitivity of synthetic vision. In FIG. 5, 51 is the ratio of rod cells that affects the sensitivity of synthetic vision when the in-field average luminance is from 0.1 Cd / m ² to 10 Cd / m ² . 52 is the ratio of rod cells affecting the sensitivity of synthetic vision when the average in-field luminance is 0.1 Cd / m ² or less. 53 is the ratio of rod cells affecting the sensitivity of synthetic vision when the average in-field luminance is 10 Cd / m ² or more. When these relationships are expressed by an equation, if the average luminance in the visual field = k (Cd / m ² ), and the ratio of rod cells affecting the sensitivity of synthetic vision = s,
s = 0 in the range of k <0.1
In the range of 0.1 <k <10, s = 1−0.5 * log k
In the range of k> 10, s = 0
It becomes.

  The ratio at which the user feels the color intensity at the time of shooting or viewing can be derived from FIG. Hereinafter, with reference to FIG. 6 and FIG. 7, the color density and the color temperature felt during shooting and viewing will be described.

FIG. 6 is a diagram illustrating a ratio of feeling the color density with respect to the assumed average luminance at the time of shooting. In FIG. 6, the horizontal axis indicates the assumed average luminance at the time of shooting, and the vertical axis indicates the ratio at which the user feels the color depth. In FIG. 6, 61 is a ratio at which the assumed average luminance at the time of photographing feels the color intensity from 0.1 Cd / m ² to 10 Cd / m ² . 62 is a ratio at which the assumed average luminance at the time of photographing senses the color density of 0.1 Cd / m ² or less. 63 is a ratio in which the assumed average luminance at the time of photographing feels the color density of 10 Cd / m ² or more.

  There are two methods for obtaining the assumed average brightness at the time of shooting, and one is a shooting parameter, which assumes the average brightness from the ISO value, aperture value, shutter speed value, and the like. In the case of a still image or a moving image, the shutter speed value is almost constant, and the ND filter multiple is also used. This calculation method will be described later.

Another method is to assume an average luminance from, for example, an APL value or a histogram as a feature amount of image data when a shooting parameter cannot be acquired from a broadcast video or the like. Here, the assumed average luminance is expressed by the following equation.
Assumed average brightness = A * (APL value * number of gradations)
A is a constant. For example, if A = 1 (Cd / m ² ) with respect to the 10-bit gradation number, an assumed average luminance of the 10-bit gradation number can be obtained. For example, when the APL value is 0.2, the number of 10-bit gradations is 1024, and the assumed average luminance is 200 Cd / m ² . This is a level corresponding to the brightness of a bright indoor or light outdoor.

  This assumed average brightness is only expected brightness and does not accurately indicate the situation at the shooting site, but it shows a trend of image data and is sufficient for this correction. Further, the constant A can be further adjusted by shaking in the range of 0.3 to 3.0. If the number of 8-bit gradations is not a 10-bit gradation number, A will be a fourfold value (about 1 to 10), and if it is a 12-bit gradation number, A will be a quarter-value (0. 1 to 1).

  The ratio at which the color density on the vertical axis is felt is determined in proportion to the ratio of the rod cells on the vertical axis in FIG. That is, corresponding to the ratios 51, 52, and 53 of the rod cells, the ratios at which the color is felt are determined as 61, 62, and 63.

FIG. 7 is a diagram showing the ratio of feeling the color density with respect to the average luminance in the visual field during viewing. In FIG. 7, the horizontal axis indicates the average luminance in the visual field at the time of viewing, and the vertical axis indicates the ratio at which the color density is felt. In FIG. 7, reference numeral 71 denotes a ratio at which the average luminance within the visual field feels the color intensity from 0.1 Cd / m ² to 10 Cd / m ² . Reference numeral 72 is a ratio at which the average luminance in the field of view senses the color density of 0.1 Cd / m ² or less. 73 is a ratio at which the average luminance within the field of view senses the color depth of 10 Cd / m ² or more. A method for obtaining the average luminance within the visual field during viewing will be described later.

  The ratio at which the color density on the vertical axis is felt is determined in proportion to the ratio of the rod cells on the vertical axis in FIG. That is, corresponding to the ratios 51, 52, and 53 of the rod cells, the ratios in which the color is felt are determined as 71, 72, and 73.

  As described above, using FIG. 6 and FIG. 7, the determination of the ratio of feeling the color intensity according to the ratio of the rod cells has been described. However, the color density is not corrected to 0 when the ratio is 1 by correcting the color density with the value of the ratio that senses the color density. Then, the place where the color disappears is made, and a sense of incongruity appears.

  In order to convert the above-mentioned ratio of feeling the color density into the correction ratio of the color density with respect to the image data, it is necessary to reduce the correction effect by applying an appropriate value within the range of 0 to 1.0 as the magnification. There is. With this constant as B, the following calculation is performed.

If the ratio of feeling the color density at the assumed average brightness at the time of shooting is X, and the ratio of feeling the color density at the average brightness in the visual field at the time of viewing is Y, the correction ratio Z for reducing the color density Is defined as follows.
When Y> X, Z = B * (Y−X)
If Y <X, Z = 0

  The reason why the correction ratio Z = 0 when Y <X is that if a photograph taken in a bright field is viewed in a dark environment, the Purkinje transition works on its own, so there is no need for correction.

  As the value of the correction ratio B, about 0.5 times is natural and desirable, and therefore, it is only necessary to select from 0.2 times where the correction effect is weak to about 0.8 times where the correction effect is strong. The higher the correction ratio B, the lighter the color density, so that the situation seen in a dark place can be reproduced in a bright environment.

Specifically, a calculation example in which the correction ratio B = 0.7 is given. For example, if the assumed average luminance at the time of shooting is 20 Cd / m ² and the average luminance in the visual field at the time of viewing is also 20 Cd / m ² , X = 1.0 from FIG. 6 and Y = 1.0 from FIG. Therefore, Z = 0.

Further, if the assumed average luminance at the time of photographing is 1 Cd / m ² and the average luminance within the visual field at the time of viewing is also 20 Cd / m ² , X = 0.5 from FIG. 6 and Y = 1.0 from FIG. Therefore, Z = 0.7 * (1.0−0.5) = 0.35. Since the correction of the color density needs only to be lighter than the correction ratio, the following equation is obtained.
Output value of color density = (1−Z) * input value of color density Accordingly, the output is performed so that the color density becomes 0.65 times.

FIG. 8 is a diagram showing a deviation of the color temperature with respect to the average luminance in the visual field. In FIG. 8, the horizontal axis indicates the average luminance in the visual field, and the vertical axis indicates the color temperature. A graph 81 in FIG. 8 is obtained by examining the color temperature of a patch that looks white with the same color temperature while changing the average luminance in the visual field. The color temperature having the same impression is obtained by changing the color temperature of the patch while changing the average luminance in the field of view, with the patch having a color temperature of 6500k as the reference white at the brightness of 10 Cd / m ² or more. .

  It was found that the color temperature gradually increased in a dark place with respect to the original color temperature of 6500 K, and that the same color temperature was felt when the color temperature was about 13000 K in a dark place. By the way, since the human sense is proportional to the reciprocal of the color temperature rather than the value of the color temperature, the use of the reciprocal of the color temperature can be controlled closer to the sense.

  Also, the correction of the color temperature is influenced by the ratio of the rod cells on the vertical axis in FIG. 5, and therefore the same ratio as the ratio of feeling the color density shown in FIGS. 6 and 7 should be used. Can do. Therefore, the reciprocal of 6500K is 0.000154, the reciprocal of 13000K is 0.0000769, and the difference between the reciprocal values is 0.000771.

When these reciprocal values are used, the corrected color temperature is obtained by the following equation.
Reciprocal of output value of color temperature = 0.000154-Z * 0.000771
As a specific numerical example, if the assumed average luminance at the time of shooting is 1 Cd / m ² and the average luminance in the visual field at the time of viewing is also 20 Cd / m ² , X = 0.5 from FIG. Since Y = 1.0 from FIG. 7, assuming that the correction ratio B = 0.7,
Z = 0.7 * (1.0-0.5) = 0.35
Reciprocal of output value of color temperature = 0.000154-0.35 * 0.000771 = 0.000127
Therefore, the output value of the color temperature is 7870K.

As described above, the necessity of correction for the fact that the color looks dark or the color temperature looks low due to the difference between the shooting environment and the viewing environment, and the value of an appropriate correction process have been described. Next, a method for obtaining the in-field average luminance will be described. The average luminance within the visual field can be obtained by the following equation.
Average luminance in the field of view = Environmental average luminance (1-Screen field ratio) + Screen average brightness * Screen field ratio

Next, how to obtain the environmental average brightness, the screen view rate, and the in-screen average brightness will be described. The average brightness (environment average brightness) of the ambient light entering the field of view from the wall surface outside the screen is obtained by the following formula.
Environmental average brightness = Screen illuminance / π * Wall reflectance

  The screen edge illuminance is measured by an illuminometer attached to the edge of the screen of the display device. The wall reflectivity is suitably set to 0.8 for white walls and about 0.5 for other walls. When the value in the actual environment is not set, it is appropriate to set the initial value to about 0.6. In addition, if the image quality (contrast) is immediately reflected in accordance with the change in the environmental average brightness, the image quality changes abruptly in a short time and becomes difficult to see. Therefore, it is effective to set a time average value of about 10 seconds. .

Next, how to obtain the screen view rate will be described. According to the following document, when the viewing angle is 10 degrees, the screen viewing rate, that is, the proportion of the screen viewing field is 57%. Since this screen view rate is a value when the screen is being watched, it is considered appropriate to drop the screen view rate to about half when the screen is not watched. In this case, it is appropriate to set the influence of external illuminance to about 0.7 and the influence of the APL value in the screen to about 0.3.
<Reference>
Herbert Grosskopf: Der Einfluss der Heligkeit-sempfindung die Bildubertragung Fernseheu, Rundfunktechnische Mitteilungen, Jg 7, Nr.4, (1963) 205-223
Note that the Japanese translation of the above document is NII-Electronic Library Service: The Institute of Image Information and Televison Engineers Vol. 19, No. 5, 55-56.

  When the angle of view of the display device that occupies the field of view is unknown, the screen field-of-view ratio, which is an influence within the screen, is arbitrarily designated between 0.3 and 0.7. In the present embodiment, the screen view rate is set to 0.4 as an intermediate provisional value.

Next, how to obtain the average luminance within the screen, that is, the APL value will be described. The in-screen average brightness is given by the following equation.
In-screen average brightness = average of linear tone values of all pixels / full tone value * peak brightness

  In addition, if the image quality (contrast) is reflected immediately in response to the change in the average brightness within the screen, the image quality changes suddenly in a short time and becomes difficult to see. Therefore, it is effective to set a time average value of about 10 seconds. is there.

The average luminance within the visual field during viewing is obtained as follows.
Average luminance in field of view = environmental luminance * (1-angle of view) + luminance in screen * angle of view The environmental luminance is obtained by measuring environmental illuminance and multiplying the reflectance of the wall. Although the reflectivity of the wall varies depending on the viewing environment, in Japan there are many whitish walls, so if it is set to 0.6, the environmental brightness = environmental illuminance * 0.6 / π.

The in-screen brightness is obtained by multiplying the APL value by the peak brightness. The peak luminance varies depending on the mode of the display device, but is, for example, 200 Cd / m ² . The angle of view is not the true screen angle but the ratio of the in-screen brightness and the off-screen brightness to the viewer. It depends on the viewing distance and screen size, and is a value of about 0.2 to 0.7. . In this embodiment, the calculation is performed as 0.3. In the case of the above numerical example, the average luminance in the visual field is as follows.
Average luminance in the field of view = ambient illuminance ** 0.6 * 0.7 / π + APL value * 200 * 0.3

FIG. 9 is a diagram illustrating a result of calculating the in-field average luminance from the screen edge illuminance and the average gradation. In FIG. 9, it is assumed that the wall surface reflectance is 0.6, the screen view rate is 0.4, and the peak luminance of the display device is 100 Cd / m ² . The screen edge illuminance takes a value from 1 to 1000 Lux, and the average gradation takes a value from 4 to 255 gradations. For example, in an environment where the screen edge illuminance corresponding to a bright living room is 100 Lux, the average luminance in the visual field in the case of bright image data with an average gradation of 128 is 20.24 Cd / m ² . Further, in an environment where the screen edge illuminance corresponding to a dark living room or a bright theater room is 30 Lux, the average luminance in the field of view when the average gradation is 16 is 3.53 Cd / m ^2. . Further, in an environment where the screen edge illuminance corresponding to a dark room or a dark theater room is 1 Lux, the average luminance in the visual field in the case of image data with an average gradation of 16 and near black is 0.21 Cd / m ^2. become.

  Next, the configuration of the display device according to the first embodiment of the present invention will be described. FIG. 10 is a diagram illustrating a configuration of the display device according to the first embodiment.

  In FIG. 10, reference numeral 101 denotes a gamma image quality adjustment circuit that performs image quality correction such as enlargement / reduction in the gamma system on input image data. Reference numeral 102 denotes an illuminance sensor. Reference numeral 103 denotes a color correction circuit that corrects the color of the image data in accordance with the average luminance within the visual field. A linear image quality adjustment circuit 104 converts the gradation of the image data output from the color correction circuit 103 into a linear system. A panel driver 105 converts image data output from the linear image quality adjustment circuit 104 into a display panel drive signal. A display panel 106 displays image data based on a display panel drive signal output from the panel driver 105. Since the input image data is generally obtained by decoding an external input or a signal received by a tuner, the block of the image input unit is not shown in FIG.

  The input image data input to the display device is gamma-based image data that normally has a gamma of 2.2, and the gamma-based image quality adjustment circuit 101 performs gamma-based image processing such as enlargement / reduction in the gamma system. . An illuminance sensor 102 is provided at the edge of the display panel 106, and the illuminance of the screen edge can be sensed. The color correction circuit 103 calculates the average luminance in the visual field from the image data output from the gamma image quality adjustment circuit 101 and the screen edge luminance from the illuminance sensor 102, and performs an appropriate color correction process based on the calculated average luminance in the visual field. Do. Since the image data output from the color correction circuit 103 is a linear system, the linear system image quality adjustment circuit 104 performs image quality adjustment such as linear edge enhancement processing.

  FIG. 11 is a diagram showing a detailed configuration of the color correction circuit 103 according to the first embodiment of the present invention. In FIG. 11, reference numeral 1111 denotes an environmental average luminance calculation unit that calculates environmental average luminance using an input from the illuminance sensor 102. Reference numeral 1112 denotes an in-screen average luminance calculation unit that calculates the in-screen average luminance from the input image data. Reference numeral 1113 denotes an averaging unit that averages the average luminance in the screen over time. Reference numeral 1114 denotes an in-field average luminance calculation unit for calculating the in-field average luminance. Reference numeral 1115 denotes a correction switching unit that switches whether to perform correction according to the average luminance in the visual field. Reference numeral 1116 denotes a luminance estimation unit that calculates the assumed average luminance at the time of shooting from the average luminance within the screen averaged over time. Reference numeral 1117 denotes a Purkinje transition amount calculation unit that calculates the Purkinje transition amount from the assumed average luminance at the time of shooting and the average luminance within the visual field. Reference numeral 1118 denotes a correction amount calculation unit that calculates a correction amount according to the Purkinje transition amount. Reference numeral 1119 denotes a color temperature changing unit that changes the color temperature based on the correction amount calculated by the correction amount calculating unit 1118. A color density changing unit 1120 changes the color density based on the correction amount calculated by the correction amount calculating unit 1118. Note that the luminance estimation unit 1116 is a configuration serving as an example of the luminance calculation unit for photographing. The visual field average luminance calculation unit 1114 is an example of the visual field luminance calculation unit. The in-screen average luminance calculation unit 1112 is an example of an in-screen luminance calculation unit.

  Based on the input from the illuminance sensor 102, the environmental average luminance calculation unit 1111 calculates the environmental average luminance within the visual field averaged over time. The in-screen average luminance calculation unit 1112 calculates the in-screen average luminance within the visual field from the input image data. Then, the in-screen average brightness calculated by the in-screen average brightness calculation unit 1112 is temporally averaged by the averaging unit 1113.

  The visual field average luminance calculation unit 1114 calculates the temporal averaged in-field average luminance using the environmental average luminance, the temporal averaged luminance in the screen, and the screen visual field ratio.

The correction switching unit 1115 stops correction when the average luminance in the field of view is 0.1 Cd / m ² or less (second threshold value), so that it is almost rod-like. The luminance estimation unit 1116 calculates the assumed average luminance from the average luminance in the screen that is temporally averaged using the assumed luminance value = A * (APL value * number of gradations).

The Purkinje transition amount calculation unit 1117 calculates Purkinje transition amount = (Y−X). X is a ratio of feeling the color density with respect to the assumed average luminance at the time of photographing shown in FIG. 6, and Y is a ratio of feeling the color density with respect to the average luminance in the field of view shown in FIG. It is. Next, the Purkinje transition amount calculation unit 1117 uses the correction ratio B to calculate the correction amount Z of the color density and the reciprocal of the output value of the color temperature.
Color density correction amount Z = B * (Y−X)
Reciprocal of output value of color temperature = Reciprocal of 6500K−Z * (Reciprocal of 6500K−Reciprocal of 13000K)

  Using the correction value Z, the color temperature changing unit 1119 performs a 3 × 3 matrix operation so as to shift the color temperature with respect to the color difference signal of the input image data. The amount of shift is set to 6500K as the original color temperature, the inverse of the color temperature output from the correction amount calculation unit 1118 is returned to the color temperature, and the matrix multiplier is determined so as to change only the difference. Since the matrix calculation for changing the color temperature is a general process, a detailed description is omitted. The color density changing unit 1120 changes the color density by applying a correction value Z to the color difference signal of the input image data.

  It should be noted that when the color density or color temperature is switched in the middle of the frame of the input image data, the display image is disturbed, so it is preferable to change the blank line between frames. In addition, when the value is suddenly switched to a large value, a feeling of interference is produced, and it is preferable that the amount of change per frame is 10K or less for the color temperature and 0.001 for the multiplier of the color depth. As a result, the maximum amount of change per second is 600K at the color temperature and 0.06 times the color density, and this level is acceptable because there is no visual discomfort.

  Up to this point, the first embodiment of the present invention has been described. However, in order to realize the objects and effects of the present invention, it is of course possible to use other embodiments having the same elements of the invention. For example, the average luminance in the field of view was calculated from the average luminance in the environment and the average luminance in the screen. Thus, it may be obtained only from the environmental average brightness.

  In the first embodiment, both the color density and the color temperature are corrected. In another embodiment, only the color density or color temperature may be corrected. In this case, the image quality can be improved by the amount corrected, and the circuit scale for performing the correction process can be reduced.

Next, a second embodiment of the present invention will be described. FIG. 12 is a diagram showing the ratio of the rod cells that influences the sensitivity of the synthetic vision of the cone cells and rod cells with respect to the average luminance in the visual field used in the second embodiment. In FIG. 12, the horizontal axis represents the average luminance within the visual field, and the vertical axis represents the ratio of rod cells that affects the sensitivity of the synthetic vision. In FIG. 12, 151 is the ratio of rod cells that affects the sensitivity of synthetic vision when the average in-field luminance is from 0.01 Cd / m ² to 100 Cd / m ² .

  If FIG. 5 is replaced with FIG. 12, the ratio of the rod cells becomes a curve, and thus FIG. 6 and FIG. 7 also become a curve. However, since these curves are obvious, they are omitted. The ratio of feeling the color density from the assumed average luminance at the time of shooting and the average luminance in the field of view is converted according to the characteristics of the curve, but other than that, it is the same as described in the first embodiment. Is the action.

Next, a third embodiment of the present invention will be described. In the third embodiment, a case where the average luminance at the time of shooting is calculated based on the shooting parameters will be described. If the shooting parameters such as ISO sensitivity, aperture, shutter speed, and ND filter multiple are known, the average luminance during shooting can be calculated. According to the CIPA DC-X004 standard, when the aperture value is F, the exposure time is T (s), and the luminance value is D (Cd / m ² ), the image plane exposure amount Hm (Lx · s) is It is given by the following equation 1.
Hm = 0.65 * B * T / F ** 2 (** is a multiplier) Equation 1
The sensitivity S is given by the following equation 2.
S = 10 / Hm Equation 2

When the expression 1 is transformed into an expression for obtaining luminance, it is expressed by the following expression 3.
D = Hm * F ** 2 / 0.65T Equation 3
Substituting Equation 2 into Equation 3 yields Equation 4 below.
D = 10 * F ** 2 / (0.65 * T * S) Equation 4
Furthermore, if the ND filter multiple is U times, it is expressed by the following equation 5.
D = U * 10 * F ** 2 / (0.65 * T * S) ... Formula 5

For example, if the ISO sensitivity value is 400, the aperture is 4, the exposure time is 1 / 250th, and the filter magnification is 1, the luminance of the proper exposure portion is 154 Cd / m ² .

Using the brightness B of the appropriate exposure portion calculated in this way, the in-screen average brightness at the time of shooting is estimated from the APL value of the screen. Since the brightness of the proper exposure portion is considered to be a median value of 0.5 in gamma 2.2, it is a position of 0.2 when converted to a linear system. Therefore, the average luminance at the time of shooting is as follows.
Average luminance at the time of shooting = D * APL value / 0.2 Equation 6

  FIG. 13 is a diagram showing a detailed configuration of the color correction circuit 103 according to the third embodiment of the present invention. In FIG. 13, reference numerals 1111 to 1115 and 1117 to 1120 are the same as those in FIG. Reference numeral 131 denotes an appropriate luminance calculation unit that calculates the luminance of the appropriate exposure portion from the shooting parameters. Reference numeral 132 denotes a photographing average luminance calculation unit that calculates photographing average luminance.

  The appropriate luminance calculation unit 131 calculates the aperture value F, the exposure time T, and the ISO value S, which are imaging parameters, according to the above equation 5, D = U * 10 * F ** 2 / (0.65 * T * S). The luminance D of the appropriate exposed part is calculated.

  The shooting average luminance calculation unit 132 calculates the shooting average luminance from the luminance D and the APL value of the appropriate exposure portion according to the above-described formula 6, shooting average luminance = D * APL value / 0.2.

  The operation of the other configuration is the same as in the first embodiment, but the Purkinje transition amount calculation unit 1117 uses the average luminance at the time of shooting instead of the assumed average luminance at the time of shooting in the first embodiment, The correction amount is calculated.

  In the first embodiment, the assumed average brightness at the time of shooting is calculated using only the APL value, but in the third embodiment, by using the shooting parameters, the more accurate average brightness at the time of shooting is used. It is possible to determine the correction amount.

  Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, a case where dark portions are strongly corrected will be described. In the first to third embodiments described above, the correction of the color density and the color temperature is performed on the entire screen. On the other hand, in the fourth embodiment, correction is not made uniformly over the entire screen, but is corrected more strongly in the dark part.

For example, as described below, the APL value obtained by taking the average of the peripheral N × N pixels of the target pixel is set as the peripheral APL value G.
G = Σ (Y value of surrounding N × N pixels) / N ** 2
If the correction amount Z ′ to be applied only to the dark part is linearly 1.5 times in the dark part and 0 times in the bright part, the above-described equation of the correction amount Z = B * (Y−X) is It is transformed as an equation. Note that the ratio of feeling the color density at the average luminance at the time of photographing is X, and the ratio of feeling the color density at the average luminance in the visual field at the time of viewing is Y.
Z '= Z * (1-G) * 1.5 = B * (Y-X) * (1-G) * 1.5
It should be noted that non-linearity can of course be similarly achieved by changing the equation to a non-linear equation.

Similarly, regarding the color temperature, it is desirable in terms of image quality that the color temperature is shifted only in the dark part. From the above equation, since the reciprocal of the output value of the color temperature is 0.000154-Z * 0.000771, the correction of only the dark portion can be strengthened by replacing Z with Z ′.
Reciprocal of output value of color temperature = 0.000154-Z ′ * 0.000771

  FIG. 14 is a diagram showing a detailed configuration of the color correction circuit 103 according to the fourth embodiment of the present invention. In FIG. 14, reference numerals 1111 to 1117 and 1119 to 1120 are the same as those in FIG. Reference numeral 141 denotes a luminance extraction unit that extracts the luminance of the region around the target pixel. Reference numeral 142 denotes a peripheral average luminance calculation unit that calculates peripheral average luminance from a plurality of extracted luminances. A correction amount calculation unit 143 calculates a correction amount from the Purkinje transition amount and the peripheral average luminance. The peripheral average luminance calculation unit 142 is an example of a peripheral region luminance calculation unit.

  The luminance extraction unit 141 includes, for example, a plurality of shift registers, and can refer to luminance values in the NxN region around the target pixel at a time. The peripheral average luminance calculation unit 142 calculates the peripheral APL value using the above formula, G = Σ (Y value of the peripheral N × N pixel) / N / N, using the Y value of the N × N region.

  In the correction amount calculation unit 143, the above formula, Z ′ = B * (Y−X) * (1−G) * 1.5, and the reciprocal of the output value of the color temperature = 0.000154-Z ′ * 0. .000771 is used to calculate the correction value of the color density and the correction value of the color temperature. By using these correction values and performing processing in the color temperature changing unit 1119 and the color density changing unit 1120 as in FIG. 1, corrected image data is obtained.

  The present invention can be applied not only to a display device but also to an image quality adjustment device inserted between a photographing device or a recording device and the display device. Hereinafter, an example thereof will be described as a fifth embodiment.

  FIG. 15 is a diagram showing a configuration of an image quality adjustment apparatus according to the fifth embodiment of the present invention. In FIG. 15, reference numeral 151 denotes a gamma image quality adjustment circuit that performs gamma image quality correction on input image data. Reference numeral 152 denotes an illuminance sensor. Reference numeral 153 denotes a color correction circuit that corrects the color of the image data in accordance with the average luminance in the visual field.

  Since the input image data input to the image quality adjusting apparatus is gamma image data to which gamma 2.2 is normally applied, the gamma image quality adjusting circuit 151 performs gamma image processing such as enlargement / reduction in the gamma system. Do. The image quality adjustment apparatus is provided with an illuminance sensor 152 and can sense environmental illuminance. The color correction circuit 153 calculates the average luminance in the visual field from the output image data from the gamma image quality adjustment circuit 151 and the illuminance value from the illuminance sensor 152, and performs appropriate color correction based on the calculated value.

  The output from the color correction circuit 153 remains a gamma system, and the output image data is connected to a general display device and displayed. The color correction circuit 153 in the fifth embodiment has the same internal configuration as the color correction circuit 103 in the first embodiment shown in FIG. As the color correction circuit 153, the color correction circuit 103 described in the second to fourth embodiments may be applied.

  In the above-described embodiment, it is possible to reproduce the change in color temperature and color density due to rod cells that occurs when the real object is viewed in a dark place on a display device that is viewed in a bright place. . In other words, according to the present embodiment, it is possible to display image data with an image quality as if actually looking at the real thing, particularly in the dark area, by generating an image that conforms to the Prokinier transition. Become.

  The image quality adjustment device according to the present embodiment can be widely used as a device connected to a display or projector using a liquid crystal, a plasma light emitting element, or an EL element, or a display device. In addition, the display device and the image quality adjustment device according to the above-described embodiments are configurations that are examples of an image processing device.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  101, 151: Gamma image quality adjustment circuit, 102, 152: Illuminance sensor, 103, 153: Color correction circuit, 104: Panel driver, 106: Display panel, 131: Appropriate luminance calculation unit, 132: Average luminance calculation unit during shooting , 141: luminance extraction unit, 142: peripheral average luminance calculation unit, 143: correction amount calculation unit, 1111: environmental average luminance calculation unit, 1112: in-screen average luminance calculation unit, 1113: averaging unit, 1114: average in field of view Luminance calculation unit, 1115: correction switching unit, 1116: luminance estimation unit, 1117: Purkinje transition amount calculation unit, 1118: correction amount calculation unit, 1119: color temperature change unit, 1120: color density change unit

Claims (7)

A photographing luminance calculation means for calculating luminance at the time of photographing of the image data displayed in the screen;
When the luminance at the time of photographing calculated by the luminance at the time of photographing is equal to or lower than a first threshold, a correction process for increasing the color temperature of the image data based on the luminance at the time of photographing and the color of the image data An image processing apparatus comprising: a correction unit that executes at least one of correction processes for reducing the darkness of the image. It further has a visual field luminance calculation means for calculating the luminance within the visual field,
2. The image according to claim 1, wherein when the luminance in the visual field calculated by the visual field luminance calculating unit is higher than a second threshold, the correction unit performs the correction process on the image data. Processing equipment. In-screen brightness calculation means for calculating brightness in the screen on which the image data is displayed,
3. The image processing according to claim 1, wherein the photographing luminance calculation unit calculates the photographing luminance based on the in-screen luminance calculated by the in-screen luminance calculation unit. apparatus.   The image processing apparatus according to claim 3, wherein the photographing luminance calculation unit further calculates the luminance at the time of photographing based on photographing parameters at the time of photographing of the image data. A peripheral area luminance calculating means for calculating the luminance of the peripheral area of the target pixel in the image data;
The image processing apparatus according to claim 1, wherein the correction unit further performs a correction process on the image data based on a luminance of a peripheral region of the target pixel. An image processing method executed by an image processing apparatus,
A shooting luminance calculation step for calculating the luminance at the time of shooting of the image data displayed in the screen;
When the luminance at the time of photographing calculated by the luminance at the time of photographing is less than or equal to a first threshold value, a correction process for increasing the color temperature of the image data based on the luminance at the time of photographing and the color of the image data And a correction step of executing at least one of correction processes for reducing the density of the image. A shooting luminance calculation step for calculating the luminance at the time of shooting of the image data displayed in the screen;
When the luminance at the time of photographing calculated by the luminance at the time of photographing is less than or equal to a first threshold value, a correction process for increasing the color temperature of the image data based on the luminance at the time of photographing and the color of the image data A program for causing a computer to execute a correction step for executing at least one of correction processes for correcting the darkness of the image.

Images