JP2012042883A - Image processing system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly control color reproduction without detecting positions of viewers based on captured images.SOLUTION: The invention includes: display information acquisition means to acquire image display surface position information of an image display device; color conversion processing means to perform color conversion processing for input image data using color conversion parameters corresponding to the image display surface position information; and output means to output the image data (after the color conversion processing) to the image display device.

Description

  The present invention relates to an apparatus for controlling color reproduction according to posture information of a display surface.

  Currently, various image display devices such as CRTs and LCDs are widely used as devices that output color images. In general, an image display device has a viewing angle characteristic in which a display color changes depending on an angle (observation angle) at which a surface (image display surface) on which an image is displayed is viewed. Therefore, for example, when the image display surface is viewed from the front and when viewed from an oblique direction, the color looks different even when the same image is output. As a method for suppressing such a change in color due to viewing angle characteristics, a technique for correcting color reproduction of an image display device according to an observation angle is known as a conventional technique. (For example, patent document 1). This correction technique first detects the light emitted from the viewer's face area or the remote control of the viewer from the image captured by the imaging means, and based on the detected position, the viewer's face with respect to the image display surface is detected. Estimate the viewing angle. And the change of the color by a viewing angle characteristic is suppressed by correct | amending according to the observation angle from which the saturation and the brightness of the image to display were acquired.

JP 2009-128381 A

  However, when the observation angle is estimated based on the detection position of the face area in the prior art, the observation angle cannot be calculated in an environment where it is difficult to detect the face area from the captured image such as a dark room. Further, when the viewer views the image with the image display device in his / her hand, there is a problem that it becomes difficult to use a remote controller for emitting light at the same time.

  Accordingly, an object of the present invention is to enable appropriate color reproduction control without detecting the position of a viewer using captured images.

  According to the first aspect of the present invention, the display information acquisition means for acquiring the posture information of the image display surface of the image display device and the color conversion parameter corresponding to the posture information of the display surface are used for the input image data. Color conversion processing means for performing color conversion processing, and output means for outputting the image data subjected to the color conversion processing to the image display device.

  The invention according to claim 4 of the present application is a display information acquisition unit that acquires posture information of an image display surface of an image display device, a color conversion processing unit that performs color conversion processing on the input image data, and the image display surface. Determining means for determining a light emission amount adjustment parameter based on the attitude information.

  According to the present invention, it is possible to appropriately control color reproduction without performing viewer position detection using captured images.

1 is a block diagram illustrating a system configuration of an image processing apparatus. FIG. 3 is a diagram illustrating a schematic functional configuration according to the first embodiment. It is a flowchart which shows the flow of an image process. 3 is a flowchart illustrating a flow of color conversion processing according to the first embodiment. It is a figure explaining the inclination of an image display surface. It is a flowchart which shows the flow of a color conversion parameter creation process. It is a figure explaining the effect of a color conversion process. It is a figure explaining the characteristic of a color conversion parameter. 4 is a diagram illustrating a color gamut corresponding to an observation angle of the output device 109. FIG. FIG. 6 is a diagram illustrating a schematic functional configuration according to a second embodiment. 3 is a flowchart illustrating a flow of output image display processing according to the first embodiment. It is a flowchart which shows the flow of light emission amount adjustment parameter creation.

Example 1
In this embodiment, it is assumed that the viewer's gazing point is at a fixed position on the image display surface (for example, the center of the image display surface), and the observation angle is estimated from the inclination of the image display surface. The image displayed on the image display surface is subjected to color conversion processing using a color conversion parameter corresponding to the viewing angle, thereby generating an image appropriately reproduced in color according to the viewing angle of the viewer. .

  First, a system configuration example of the image processing apparatus according to the present embodiment will be described with reference to FIG. The CPU 101 uses the RAM 102 as a work memory, executes programs stored in the ROM 103 and the hard disk drive (HDD) 105, and controls each component described later via the system bus 112. Thereby, various processes described later are executed. The HDD interface (I / F) 104 is an interface such as serial ATA (SATA) for connecting a secondary storage device such as the HDD 105 or an optical disk drive. The CPU 101 can read data from the HDD 105 and write data to the HDD 105 via the HDD I / F 104. Further, the CPU 101 can expand the data stored in the HDD 105 in the RAM 102 and similarly store the data expanded in the RAM 102 in the HDD 105. The CPU 101 can execute the data expanded in the RAM 102 as a program. An input interface (I / F) 106 is a serial bus interface such as USB or IEEE 1394 for connecting an input device 107 such as a keyboard, a mouse, a digital camera, and a scanner. The CPU 101 can read data from the input device 107 via the input I / F 106. An output interface (I / F) 108 is a video output interface such as DVI or HDMI for connecting an output device 109 such as an image display device. The CPU 101 can send data to the output device 109 via the output I / F 108 to execute display. The attitude detection interface (I / F) 110 is a serial bus interface such as USB or IEEE 1394 for connecting an attitude detection device 111 such as an acceleration sensor or an angular velocity sensor. The posture detection device 111 is attached to the image display surface of the output device 109, and the CPU 101 can read posture information on the image display surface from the posture detection device 111 via the posture detection I / F 110.

  Next, an outline functional configuration when performing a series of processes according to the present embodiment will be described with reference to FIG. The functions of the units 201 to 204 in FIG. 2 are realized by the CPU 101 executing a program stored in advance.

  The image input unit 201 acquires the input image data 205 from the input device 107 or a storage device such as the ROM 103 or the HDD 105. The display parameter input unit 202 acquires a display parameter 206 including posture information on the image display surface from the posture detection device 111. The image processing unit 203 performs color conversion on the input image data 205 based on the display parameter 206 to generate output image data 207. The image output unit 204 outputs the output image data 207 to the output device 109.

  FIG. 3 is a flowchart showing an operation procedure of a series of processes in the image processing apparatus. After a computer-executable program describing the procedure shown in the flowchart of FIG. 3 is read from the ROM 103 or HDD 105 onto the RAM 102, the CPU 101 executes the program to execute the process.

  First, in step S301, the image input unit 201 acquires input image data 205 captured by an input device 107 such as a digital camera. The image input unit 201 may acquire input image data from the input device 107, or the image input unit 201 may read input image data stored in a storage device such as the ROM 103 or the HDD 105.

  In step S <b> 302, the display parameter input unit 202 acquires a display parameter 206 including posture information of the image display surface from the posture detection device 111. Here, the orientation information of the image display surface is the inclination θ of the image display surface from the reference state O. FIG. 5 shows an example of the inclination θ of the image display surface. In the example of FIG. 5, the state where the image display surface is installed so as to face the viewer C (the state indicated by the oblique lines in the figure) is the reference state O, but the viewer holds the image display surface in a natural posture. Other states, such as a state held in the case, may be used as the reference state. In the example shown in the figure, the angle with the axis Y as the center of rotation is the inclination θ of the image display surface, but another axis may be the center of rotation. The acquired display parameter 206 is stored in the RAM 103 or the like.

  In step S303, the image processing unit 203 performs color conversion processing on the input image data 205 using the color conversion parameter corresponding to the display parameter 206 acquired in step S302, and generates output image data 207. Assuming that the viewer's viewpoint position and the viewer's gazing point are at fixed positions on the image display surface, the viewer's viewing angle with respect to the image display surface is considered to be equal to the inclination of the image display surface. Therefore, in this embodiment, the color conversion parameter used for the color conversion process is determined by using the inclination θ of the image display surface included in the display parameter 206 as the observation angle θ.

  Details of the color conversion process performed by the image processing unit 203 will be described below with reference to the flowchart shown in FIG. First, in step S401, a color conversion parameter corresponding to the observation angle θ is determined based on a plurality of color conversion parameters stored in the storage device in association with the observation angle. In this embodiment, a color conversion LUT is used as a color conversion parameter.

  Here, the observation angle θ is the inclination θ of the image display surface included in the display parameter 206 acquired in step S302. Details of the method of creating the color conversion parameter corresponding to the observation angle will be described later. Here, when there is no color conversion parameter corresponding to the observation angle θ, the color conversion parameter corresponding to the two observation angles θ0 and θ1 satisfying θ0 <θ <θ1 is used to correspond to the observation angle θ using interpolation processing. Generate color conversion parameters. In addition, when there is no color conversion parameter corresponding to the observation angles θ0 and θ1, the color conversion parameter whose associated observation angle is closest to θ is selected.

  For example, it is assumed that conversion parameters associated with observation angles in increments of 5 ° from −70 ° to 70 ° are held in the storage device. At this time, if the inclination θ of the image display surface included in the display parameter 206 is 20 °, a color conversion parameter corresponding to θ = 20 ° is selected. When θ is 22 °, a color conversion parameter corresponding to θ = 22 ° is generated by interpolation processing from the color conversion parameters corresponding to θ0 = 20 ° and θ1 = 25 °. When θ is 80 °, the color conversion parameter associated with θ = 70 ° is selected. Note that the range and interval of the corresponding observation angles when preparing the color conversion parameters in advance are not limited to the example described above, and may be wider or narrower. In addition, the upper and lower limits of the observation angle range may not be symmetric about 0 °, and the observation angle intervals may be non-uniform.

  In step S402, each pixel data of the input image data 205 is converted using the color conversion parameter determined in step S401, and output image data 207 is generated. If there is an RGB value not described in the color conversion LUT in the input image 205, the pixel value of the output image may be obtained from the neighboring grid points using interpolation processing such as tetrahedral interpolation. The color conversion parameter stored in association with the observation angle suppresses a change in color due to the viewing angle characteristic, so that the display color when the image display surface is viewed at a certain reference angle θr is different from other observation angles. Create it so that it will be reproduced when viewed. Color conversion parameters are created based on the characteristics of the image display device in the reference posture and the characteristics of the image display device in the creation target posture.

  Hereinafter, a method of creating a color conversion parameter corresponding to the observation angle will be described with reference to the flowchart of FIG. However, in this embodiment, the color conversion parameter is a three-dimensional color conversion LUT in which a correspondence relationship for converting an RGB value indicating input image data into a device RGB value of the image display apparatus (output device 109) is described.

  First, in step S <b> 601, RGB values corresponding to the respective grid points generated by dividing the device RGB color space depending on the output device 109 into grids are output to the output device 109. Then, the XYZ measurement data Xr and X obtained by measuring the image displayed on the output device 109 from each of the two directions of the reference angle θr and the observation angle θ are associated with the device RGB values of the lattice points.

  In step S602, the XYZ values of the measurement data Xr and X associated with the device RGB values in step S601 are converted into L * a * b * values using the following formulas, respectively, and the reference angle θr and the observation angle θ respectively. L * a * b * data Pr and P are obtained.

  In step S603, first, the color gamut at the observation angle θ of the output device 109 from the L * a * b * data P of the observation angle θ calculated from the XYZ values of the measurement data Xr associated with the device RGB values in S602. Is calculated. Next, in step S601, the L * a * b * data Pr of the reference angle θr associated with the device RGB value is mapped in the color gamut at the observation angle θ of the output device 109, and L * after the color gamut compression processing is performed. a * b * data P ′ is obtained. FIG. 9 is a diagram showing a color gamut according to the observation angle of the output device 109, and shows an L * -a * cross section in the CIELAB space of the color gamut of the output device 109. A line 91 is a color gamut when the viewer faces the image display surface. The observation angle of the color gamut indicated by the dotted line 92 is closer to the opposite angle than the observation angle of the color gamut indicated by the broken line 93. The color gamut visible to the viewer becomes narrower as the viewing angle of the output device 109 deviates from the front. In the color compression processing, the L * a * b * value outside the color gamut at the observation angle θ but within the color gamut at the reference angle θr is changed to the L * a * b * value in the color gamut at the observation angle θ. It is a process to convert. There are various methods for color gamut compression. For example, as a method of visual matching, colors in the color gamut are reproduced faithfully without being compressed as much as possible, and colors outside the color gamut are compressed to the high saturation part in the color gamut. Therefore, it is preferable to use a method for maintaining gradation.

  In step S604, the L * a * b * data P ′ after the color gamut compression processing is inversely converted into XYZ values. In step S605, the device RGB value corresponding to the XYZ value of P ′ is obtained using the correspondence relationship between the device RGB value created in step S601 and X.

  In step S606, the device RGB value obtained in step S605 is associated with the device RGB value corresponding to the L * a * b * data Pr before the color gamut compression processing.

  The processing of steps S602 to S606 is performed for each of the XYZ values of the measurement data Xr associated with the RGB values corresponding to the respective grid points generated by dividing the device RGB color space depending on the output device 109 into a grid. As a result, a color conversion LUT corresponding to the observation angle θ is created.

  In the present embodiment, a plurality of color conversion LUTs are created in advance by performing the processes in steps S701 to S706 described above for various observation angles, and are stored in the storage device.

  The effect of the color conversion process described above will be described with reference to FIG. FIGS. 7A and 7B show examples of output images generated by the above-described color conversion processing according to observation angles θr and θ (where θr is a reference angle and θr ≠ θ), respectively. FIG. 7D shows an appearance when the image display surface displaying the output image corresponding to the observation angle θr (FIG. 7A) is viewed from the observation angle = θr. When the viewing angle of the viewer changes to an arbitrary θ, it is expected that the displayed image appears in the same color as when viewed from the viewing angle = θr as shown in FIG. However, when the image display surface on which the output image corresponding to the observation angle θr (FIG. 7A) is displayed is viewed from the observation angle = θ (θ ≠ θr), the image display device has a viewing angle characteristic. As shown in FIG. 7 (e), the colors look different. On the other hand, when the output image (FIG. 7B) generated by correcting the color reproduction according to the observation angle θ is displayed, when the image display surface is viewed from the observation angle θ °, FIG. The image can be viewed in the color shown in f).

  As described with reference to FIG. 9, since the color gamut changes depending on the observation angle, there are some colors that cannot be faithfully reproduced. Usually, the color gamut when the image display surface is viewed from an angle is narrower than the color gamut when viewed from the front, so the image color when viewed from the front is the same color when viewed from an angle. It cannot be reproduced. In this embodiment, since color compression processing is performed, colors that cannot be faithfully reproduced can be reproduced so as to be close to each other in colors that can be reproduced.

  On the other hand, by setting the reference angle θr so that the absolute value thereof is a large value, it is possible to indicate that the same color always appears regardless of the observation angle. For example, when the reference angle θr is set to the observation angle of the color gamut of the dotted line 93 in FIG. 9, the color gamut of the line 71 and the dotted line 72 is larger than the color gamut of the dotted line 93. Can faithfully reproduce the color observed from the observation angle of the broken line 93. However, in this case, the wide color gamut inherent in the image display device, which can be reproduced when the image display surface is viewed from the front, cannot be fully utilized.

  Therefore, when the color reproduction accuracy of the output image generated in step S402 is out of a predetermined allowable range with respect to the color when viewed from the reference angle θr, a notification to that effect is given to the viewer. May be. With this configuration, when the reference angle θr is adjusted, an image is displayed utilizing the original color gamut of the image display device, and when a certain color reproducibility cannot be maintained according to the observation angle θ, viewing is performed. It is possible to inform the user of a decrease in color reproducibility. As a result, the viewer can view a high-brightness and high-saturation image using the original color gamut of the image display device while changing the observation angle, and if the color reproducibility of the display image decreases, this is indicated. The effect is that it can be accurately grasped. As a notification method of color reproducibility degradation, for example, it is conceivable to generate an output image in which brightness and saturation are explicitly lowered in step S402. FIGS. 7C and 7G show examples of the appearance when the output image 207 in this case and the image display surface displaying the output image 207 are viewed from the corresponding observation angles. In addition to this method, a marker indicating that the color reproducibility has deteriorated may be displayed on the display screen * 2.

  As an index of color reproduction accuracy, an average color difference of display colors and a color gamut volume ratio when the image display surface is viewed from each angle of the reference angle θr and the observation angle θ can be used. These index values may be calculated in step S402 every time the observation angle θ changes. Alternatively, a value calculated in advance for each observation angle may be included in the display parameter and stored. At this time, for an observation angle whose color reproduction accuracy is known to be outside the allowable range, a color conversion LUT is created by reducing the lightness and saturation of the target color during the color gamut compression processing in step S603 described above. And may be memorized.

  In addition, if the amount of decrease in lightness and saturation is increased as the absolute value of the observation angle θ increases, when the observation angle exceeds a certain value, the image display surface naturally darkens as it becomes oblique. It is also possible.

  The characteristics of the color conversion LUT obtained by the above processing will be described with reference to FIG. A device RGB value of an image display apparatus obtained by performing color conversion processing on a certain input RGB value v using a color conversion LUT created for an observation angle θ is defined as vθ. The solid and broken lines in FIGS. 8A, 8B, and 8C output v or vθ on the image display surface, respectively, and lightness L, saturation S, and hue H when viewed from the front (θ = 0). An example of At this time, the solid line and the broken line in FIGS. 8D, 8E, and 8F indicate the lightness L, saturation S, and S when the image display surface that outputs v or vθ is viewed in a state tilted by θ, respectively. An example of hue H is shown.

  In the case of the example shown in FIG. 8, the lightness L when the image display surface that outputs v is tilted by θ is tilted from the front as shown by the broken line in FIG. 8D due to the viewing angle characteristics of the image display surface. As the value increases, the value decreases. On the other hand, the color conversion LUT works so as to cancel this decrease in θ that can reproduce the color at the reference angle θr. That is, the color conversion LUT has a characteristic of converting v into vθ so that the brightness is constant no matter where it is viewed as indicated by the solid line. At this time, the brightness when vθ is output and viewed from the front increases as the inclination increases from the front, as shown by the solid line in FIG. Further, for θ incapable of reproducing the color at the reference angle θr, for example, as shown by the solid line in FIG. 8D, the reference lightness Lr is converted to vθ so that the lightness is smoothly lowered.

  The output image data 207 generated by the processing from step S401 to step S402 is stored in the ROM 103 or the HDD 105 when the capacity is large. Finally, in step S304, the image display unit 204 outputs the output image 207 to the output device 109, and displays the output image 207 on the image display surface.

  By performing the processing control described above, it is possible to appropriately correct the color reproduction of the image display device in accordance with the viewing angle of the viewer. In the present embodiment, the three-dimensional color conversion LUT is used as the color conversion parameter. However, the same effect can be obtained even if the conversion is performed using a conversion matrix or a conversion function that associates the output color signal with the input color signal. be able to.

(Example 2)
In the first embodiment, the method for correcting the color reproduction of the image display device according to the observation angle by performing color conversion processing using different color conversion LUTs according to the inclination of the image display surface has been described. In this embodiment, the color reproduction correction according to the observation angle is realized by controlling the light emission amount of the output device according to the inclination of the image display surface.

  An outline functional configuration when performing a series of processes in the second embodiment will be described with reference to FIG. About the image input part 201 and the display parameter input part 202, since it is the same as the content demonstrated using FIG. 2 in Example 1, it abbreviate | omits description. Hereinafter, details of the image processing unit 901 and the image display unit 904 different from those of the first embodiment will be described. The image processing unit 901 converts each pixel value of the input image 205 into a device RGB value of the image display device, and generates an output image 207. The image display unit 902 controls the light emission amount of the output device based on the display parameter 206 and displays the output image 207 on the output device 109.

  The details of the processing will be described below using the flowchart shown in FIG. 3 as in the first embodiment. However, since the input image acquisition process in step S301 and the display parameter acquisition process in step S302 are the same as those in the first embodiment, description thereof is omitted.

  In step S <b> 303, the image processing unit 203 performs color conversion processing on the input image 205 to generate an output image 207. At this time, the color conversion parameters used for the color conversion processing are the same regardless of the display parameters unlike the first embodiment. For example, a color conversion LUT when the observation angle θ is 0 ° is created in advance and used. Note that the display parameter acquisition process in step S302 and the color conversion process in step S303 may be switched in order.

  In step S304, the image display unit 204 selects a device control parameter based on the display parameter 206 acquired in step S302. Then, by outputting the selected device control parameter and the output image 207 to the output device 109, the output device 109 is controlled according to the observation angle, and the output image 207 is displayed on the image display surface. Here, as in the first embodiment, the device control parameter is determined using the inclination θ of the image display surface included in the display parameter 206 as the observation angle θ. Details of the output image display process will be described below using the flowchart shown in FIG.

  First, in step S1101, various observation angles are created in advance, and device control corresponding to the observation angle θ is selected from a plurality of device control parameters associated with the angles and stored in a storage device such as the ROM 103 or the HDD 105. Determine the parameters. As a specific selection method, the same selection process can be applied by replacing the color conversion parameter with the device control parameter in the color conversion parameter selection process in step S401 described in the first embodiment. As in the first embodiment, the device control parameters stored in the storage device in advance are reproduced so that the display color when the image display surface is viewed at the reference angle θr is also viewed from other observation angles. create.

  Hereinafter, a method for creating device control parameters will be described with reference to the flowchart of FIG. However, in this embodiment, the device control parameter is a light emission amount adjustment parameter for adjusting the light emission amount of the output device 109. For example, a voltage value that determines the light emission amount of the light emitting element is used as the light emission amount adjustment parameter.

  First, in step S1201, the luminance with respect to the reference angle θr is measured. Specifically, the luminance is measured with the image display surface tilted by θr with respect to the luminance measuring machine. At this time, the measurement is performed on the measurement region set in the image display surface. However, in this embodiment, the measurement area is the center of the image display surface on which the viewer is gazing. As described above, the luminance value with respect to the reference angle θr is acquired.

  In step S1202, a light emission amount adjustment parameter with respect to the observation angle θ is acquired. Specifically, the measurement region is measured in a state where the image display surface is inclined by θ with respect to the luminance measuring machine. Then, the light emission amount adjustment parameter is adjusted so that the measured luminance value becomes equal to the luminance value with respect to the reference angle θr acquired in step S1201. The light emission amount adjustment parameter when the luminance value is equal to the luminance value with respect to the reference angle θr is acquired as the light emission amount adjustment parameter with respect to the observation angle θ.

  Finally, in step S1203, the light emission amount adjustment parameter acquired in S1202 is stored in the storage device in association with the corresponding observation angle θ.

  In the present embodiment, the processing from step S902 to step S903 is performed for various observation angles, so that a plurality of light emission amount adjustment parameters are associated with the observation angles and stored in the storage device in advance. As in the first embodiment, when the color reproduction accuracy of the output image 207 is out of a predetermined allowable range, it may be configured to notify the viewer to that effect. As a notification method of color reproducibility reduction, for example, it is conceivable to explicitly select a light emission amount control parameter with a small light emission amount in step S1101.

  In step S1102, the device control parameter and output image 207 obtained in step S1101 are output to the output device 109, and the output image 207 is displayed with the light emission amount corresponding to the display parameter 206 according to the device control parameter.

  By performing the processing control described above, it is possible to appropriately correct the brightness of the image display device according to the viewing angle of the viewer.

Claims (7)

Display information acquisition means for acquiring posture information of the image display surface of the image display device;
Color conversion processing means for performing color conversion processing on input image data using color conversion parameters according to the orientation information of the display surface;
An image processing apparatus comprising: output means for outputting the color-converted image data to the image display apparatus.   The image processing apparatus according to claim 1, wherein the color conversion parameter is created based on characteristics of the image display apparatus in a reference attitude and characteristics of the image display apparatus in the creation target attitude. .   The image processing apparatus according to claim 1, further comprising a notifying unit that notifies a user when the color reproduction accuracy of the result of the color conversion processing unit is outside a predetermined allowable range. Display information acquisition means for acquiring posture information of the image display surface of the image display device;
Color conversion processing means for performing color conversion processing on the input image data;
An image processing apparatus comprising: a determination unit that determines a light emission amount adjustment parameter based on posture information of the image display surface. A display information acquisition step of acquiring posture information of the image display surface of the image display device;
A color conversion processing step for performing color conversion processing on input image data using color conversion parameters according to the orientation information of the display surface;
And an output step of outputting the color-converted image data to the image display device. A display information acquisition step of acquiring posture information of the image display surface of the image display device;
A color conversion processing step for performing color conversion processing on the input image data;
A determination step of determining a light emission amount adjustment parameter based on the posture information of the image display surface.   A program for causing a computer to function as the image processing method according to claim 5 or 6.

Images