JP2018005054A - Image processing device and method for controlling the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image to which proper gradation conversion processing is performed by an individual user.SOLUTION: An image processing device for performing gradation conversion processing to an input image includes: display control means for displaying a test image including images of a plurality of color charts photographed under first illuminance and images of a plurality of color charts photographed under second illuminance on a display part; reception means for receiving the input of a perception value related to brightness with respect to each color chart in the test image displayed on the display part; decision means for deciding a profile used for the gradation conversion processing on the basis of the perception value with respect to each color chart received by the reception means; and correction means for performing the gradation conversion processing to the input image by using the profile decided by the decision means, to generate a correction image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image gradation conversion technique.

  In recent years, images with a wide dynamic range (hereinafter referred to as “D range”) have become easier to acquire due to improved performance of an imaging system such as a digital camera. On the other hand, the wide D range of the display system has not progressed as much as the imaging system. Therefore, it is necessary to perform gradation conversion on the input image so that the wide D range of the input image is within the D range of the display system.

  In order to output an image preferable for the user, Patent Document 1 proposes a method of correcting an image with a color profile corresponding to user information (residential area, age, etc.). Patent Document 2 proposes a technique for generating a plurality of corrected images (a plurality of combined result images and gradation corrected images) from an input image, and determining and displaying a priority order based on information provided by a user. Yes.

JP 2003-296723 A JP 2012-119761 A

  In order to realize a gradation conversion method that outputs an image preferable for the user, it is necessary to consider how the user who needs the image perceives brightness. However, the brightness perceived by the user varies among individuals, and an object having a large scene brightness value is not always perceived brightly. Therefore, even if tone conversion is performed at the same level regardless of the user, an image preferable for the user is not always provided.

  The present invention has been made in view of such problems, and an object of the present invention is to be able to provide an image subjected to appropriate gradation conversion processing for each user.

In order to solve the above-described problems, an image processing apparatus according to the present invention has the following configuration. That is, an image processing apparatus that performs gradation conversion processing on an input image
A test image including a plurality of color chart images photographed under a first illuminance and a plurality of color chart images photographed under a second illuminance darker than the first illuminance is displayed on a display unit. Display control means for
Receiving means for receiving an input of a perceptual value related to brightness for each color chart in the test image displayed on the display unit;
Determining means for determining a profile to be used for the gradation conversion processing based on a perceptual value for each color chart received by the receiving means;
Correction means for performing a gradation conversion process on the input image using the profile determined by the determination means to generate a corrected image;
Have

  ADVANTAGE OF THE INVENTION According to this invention, the image which performed the suitable gradation conversion process by each user can be provided.

It is a figure which shows the structural example of an image processing apparatus. It is a flowchart which shows the process of an image processing apparatus. It is a figure which shows an example of a test image. It is a figure which shows an example of UI display in the brightness order designation | designated test in 1st Embodiment. It is a figure which shows the structural example of the profile change part in 1st Embodiment. It is a flowchart which shows the process of the profile change part in 1st Embodiment. It is a flowchart which shows the process of the image correction part in 1st Embodiment. It is a figure which shows an example of UI display in a brightness | luminance point designation | designated test. It is a figure which shows an example of a brightness point designation | designated test result. It is a figure which shows an example of a gradation conversion profile. It is a figure which shows the example of selection of the gradation conversion profile stored in profile DB. It is a figure which shows the structural example of the profile change part in 2nd Embodiment. It is a flowchart which shows the process of the profile change part in 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments are merely examples, and are not intended to limit the scope of the present invention.

(First embodiment)
As an image processing apparatus according to a first embodiment of the present invention, an image processing apparatus that generates a gradation conversion profile and performs gradation conversion processing (dynamic range conversion processing) will be described below as an example. In particular, a description will be given of a mode of selecting a profile stored in advance in a profile database (DB) based on the brightness rank designation test result for the user.

<Overview>
The relationship between the magnitude of the luminance value and the brightness perceived when actually viewed is different for each user and is not necessarily a linear relationship. Therefore, the magnitude relationship between the magnitude of the luminance value and the perceived brightness may be reversed. Therefore, a test is performed on the user to determine whether or not the brightness perception and the pixel value are reversed between the objects under the bright region and the dark region. For a user whose brightness perception tendency is reversed, different gradation conversion is performed for each image area corresponding to the brightness.

<Device configuration>
FIG. 1 is a block diagram illustrating a configuration example of the image processing apparatus according to the first embodiment.

  The image data input unit 103 is a functional unit that inputs image data (input image data) of an input image. The input image is a still image represented by RAW data at the time of shooting, and a still image obtained by performing predetermined image processing (for example, noise removal processing or demosaicing processing) on the still image represented by RAW data. It is. The input image may be a still image as a frame image constituting moving image data. The input image data is supplied to the image correction unit 106 via the image data input unit 103.

  The brightness designation unit 104 is a functional unit that inputs a brightness result from a test to be described later to the profile change unit 105. The profile changing unit 105 is a functional unit that supplies a gradation conversion profile (hereinafter referred to as profile) used for correction to the image correcting unit 106 according to the result input from the brightness specifying unit 104.

  The image correction unit 106 is a functional unit that performs gradation conversion processing based on the profile supplied from the profile changing unit 105. Specifically, the input image data is converted into corrected image data having a narrower dynamic range. The generated corrected image data is supplied to the image data output unit 107.

  The image data output unit 107 is a functional unit that outputs image data to the display device 108 that is a display unit such as a liquid crystal display and performs display control. Although details will be described later, the input test image data and corrected image data are output to the display device 108. The test image data will be described later with reference to FIG.

<Operation of the device>
FIG. 2 is a flowchart for explaining processing of the image processing apparatus 101.

  In step S <b> 1, the image data output unit 107 supplies the test image data 102 to the display device 108. The test image data 102 is image data having a known pixel value including a test image obtained by photographing a color chart having a plurality of reflectances under at least two different illuminances.

  FIG. 3 is a diagram illustrating an example of a test image used in step S1. The test image 31 is an image obtained by photographing a bright area 33 and a dark area 34 which are two different illuminance areas. In each of the bright region 33 and the dark region 34, a chart including a plurality of color charts is provided. Here, the charts 35 and 36 installed in the bright area 33 and the dark area 34 are the same. Specifically, it is assumed that the same material and shape are used, and a chart composed of white, gray, and black color charts is used. Note that the chart is not limited to the above-described chart, and any chart can be used as long as it can determine whether or not the brightness perception and the pixel value are reversed with respect to the object under the bright area and the dark area as described later. Is available.

  Further, here, the bright area 33 and the dark area 34 are created by creating a virtual room in the studio and installing illumination outside the window 32 provided in the room, and illuminating the room only with light inserted from the window 32. However, it may be realized by other methods.

  In step S2, the brightness designation unit 104 inputs a brightness result that is a perceptual value of the user. This brightness result is derived from a test for the user described later with reference to FIG.

  In step S <b> 3, the profile changing unit 105 selects a profile corresponding to the brightness result from the profiles generated in advance based on the derived brightness result, and supplies the selected result to the image correction unit 106. Details of profile selection will be described later with reference to FIG. Details of the profile generated in advance will be described later with reference to FIGS.

  In step S <b> 4, the image data input unit 103 acquires input image data and supplies it to the image correction unit 106. In step S5, the image correction unit 106 corrects the input image data supplied in S4 according to the profile supplied in S3. The operation of the image correction unit 106 will be described later with reference to FIG. In step S <b> 6, the image data output unit 107 outputs the corrected image data generated in S <b> 5 to the display device 108.

<Details of brightness result specification (S2)>
The designation of the brightness result in S2 described above accepts user responses to the eight color charts in the test image shown in FIG. This test for the user is aimed at obtaining how the user perceives brightness. Hereinafter, the test in S2 is referred to as “brightness order designation test”. The test contents for the user will be described with reference to FIG.

  FIG. 4 is a diagram illustrating an example of a UI for inputting a user response to the brightness order designation test. In the area 41, the test image 31 is displayed. The user designates the brightness order perceived by the user for each color chart of the charts 35 and 36 in the test image 31 displayed by the display device 108. Specifically, the user refers to the eight color charts in the test image 31 and inputs the relative brightness ranking perceived by the user into the dialog box 42.

  In this case, the brightness rank of “1” for the color chart A (white 1) in the bright area 33 and the “8” rank for the color chart H (black 2) in the dark area 34. Is set in advance. That is, the user designates any of “2” to “7” for the remaining six color charts.

  When the input completion button 43 is selected by the user, the brightness specifying unit 104 supplies the received brightness result to the profile changing unit 105. As will be described later, the operation of the profile changing unit 105 branches depending on whether the brightness rank (order) of each color chart and the order of the pixel values of each color chart match in the brightness order designation test. It will be.

<Details of profile change (S3)>
FIG. 5 is a diagram illustrating an example of the configuration of the profile changing unit 105 according to the first embodiment.

  The brightness order determination unit 201 determines whether or not there is a reversal between the brightness order of each color chart designated by the brightness designation section 104 and the order of the pixel values of each color chart. The profile selection unit 202 selects a corresponding profile from the profile DB 203 based on a combination of information on “used test image” and the above-mentioned “presence / absence of reversal”. The profile DB 203 stores in advance a profile generated by a generation method described later together with the test image 31. The profiles stored in the profile DB 203 will be described later with reference to FIGS.

  FIG. 6 is a flowchart for explaining the processing of the profile changing unit 105.

  In step S21, the brightness order determination unit 201 determines the brightness order for each color chart input in the dialog box 42 as a brightness order designation test result.

  In step S22, the brightness order determination unit 201 determines whether or not the designated brightness order matches the order of the pixel values of each color chart. That is, it is determined whether or not the order has been reversed, and information on “presence / absence of reversal” and “used test image” is supplied to the profile selection unit 202. If the order is not reversed, the process proceeds to step S23. If the order is reversed, the process proceeds to step S24.

  In step S23, the profile selection unit 202 selects a single profile prepared in the profile DB 203 in advance. On the other hand, in step S24, the profile selection unit 202 selects a plurality of profiles prepared in advance in the profile DB 203. The profile selected in S23 and S24 is determined by a combination of information on “used test image” and “presence / absence of reversal”.

  In step S <b> 25, the profile selection unit 202 supplies the selected profile to the image correction unit 106. Then, the image correction unit 106 performs gradation conversion on the image based on the supplied profile.

<Details of Image Correction (S5)>
FIG. 7 is a flowchart showing processing in the image correction unit 106.

  In step S <b> 31, the image data input unit 103 supplies the input image data to the image correction unit 106.

  In step S32, the image correcting unit 106 determines whether the profile supplied from the profile changing unit 105 in S25 is single or plural. If the supplied profile is single, the process proceeds to S33, and if it is plural, the process proceeds to S34.

  In step S33, the image correction unit 106 corrects the input image according to the single profile supplied in S25.

  In step S34, the image correction unit 106 divides the subject area of the input image data into a plurality of brightness areas according to the size (lightness) of the pixel value. For example, it is divided into two areas, a bright area and a dark area. Any known method can be used as a method for dividing the subject area. For example, object recognition using learning data by a neural network, a method using a histogram of pixel values, a method described in JP-A-2006-39666, or the like can be used. Also, the determination method of the subject area may be a method of determining for each block divided by an arbitrary size, not for each pixel.

  In step S35, each area of the input image divided in step S34 is corrected according to a plurality of profiles corresponding to each area supplied from the profile changing unit 105. As a correction method, a lookup table (LUT) method used in general color management can be used.

  After performing the correction in S33 or S35, the image correction unit 106 supplies the corrected image data to the image data output unit 107.

<Tone conversion profile>
Hereinafter, a method of generating a profile that is selected in the above-described S23 and used for correction in S33 or S35 will be described. The generated profile is stored in the profile DB 203. Although details will be described later, a test (brightness point specification test) that specifies the brightness perceived for each color chart in the test image in advance as a brightness score is performed, and a tone conversion curve derived from the test result is obtained. Generate a profile based on it. The subject of this preliminary test is referred to as a preliminary profile creator.

  Here, the brightness score is a relative sensation value of brightness designated by the pre-profile creator for each color chart when observing each color chart. Since there are individual differences in the brightness points designated for each color chart by the pre-profile creator, a tone conversion curve taking into account the individual differences is obtained from the test results.

  FIG. 8 is a diagram illustrating an example of a UI for inputting a response of the pre-profile creator to the brightness point designation test. In the area 51, the test image 31 is displayed. The pre-profile creator designates a brightness value for each color chart of the charts 35 and 36 in the test image 31 displayed by the display device 108. Specifically, the pre-profile creator refers to the eight color charts in the test image 31 and inputs relative brightness values to the dialog box 52.

  Here, the brightness of the color chart A (white 1) in the bright area 33 is “100” points, and the color chart H (black 2) in the dark area 34 is “0” points. Assume that the value is preset. That is, the pre-profile creator designates any of “0” points to “100” points for the remaining six color charts.

  When the input completion button 53 is selected by the pre-profile creator, the brightness designation unit 104 supplies the brightness result to the profile change unit 105.

  FIG. 9 is a diagram illustrating an example of the brightness point designation test result. The horizontal axis is the pixel value in the test image 31, and the vertical axis is the brightness score designated by the pre-profile creator. However, both the horizontal axis and the vertical axis are shown as normalized values with the maximum value being 1. White circles indicate the results for the color charts (that is, color charts A to D) in the chart 35, and black circles indicate the results for the color charts in the chart 36 (that is, color charts E to H). Here, it is assumed that the pixel values of the color charts A to H are A> B> C> D> E> F> G> H.

  FIG. 9A exemplarily shows a case where the order of the pixel values of each color chart in the test image 31 matches the order of the brightness points designated by the pre-profile creator. That is, the brightness points of the color charts A to H are A> B> C> D> E> F> G> H. In this case, as shown in FIG. 9A, the discrete points corresponding to the eight color charts are approximated by a single curve. As the approximation method, any known method such as gamma approximation or logarithmic approximation can be used. That is, a single gradation conversion curve can be determined by approximating the brightness point designation test results for all color charts (here, 8 color charts) with a single approximate curve.

  FIG. 9B shows a case where the order of pixel values of each color chart in the test image 31 and the order of the brightness points designated by the pre-profile creator do not match (that is, inversion occurs). Illustratively. Here, the brightness scores of the color charts A to H are A> B> C> E> D> F> G> H. In this case, it is determined that there is a tendency to generate a gap between the actual luminance value of the display image and the brightness perceived by the pre-profile creator, and a gradation conversion curve is generated for each illuminance region. For example, as shown in FIG. 9B, discrete points corresponding to eight color charts are approximated by two approximate curves.

  In FIG. 9B, the point 11 shows the result of the brightness point designation test for the color chart E (white 2) in the chart 36, and the point 12 shows the result for the color chart D (black 1) in the chart 35, respectively. Show. That is, although the pixel value of the color chart D (black 1) is a larger value (bright value) than the pixel value of the color chart E (white 2), the pre-profile creator creates the color chart D (black). This indicates that the color chart E (white 2) is designated with a larger brightness score than 1).

  When reverse rotation occurs as shown in FIG. 9B, for each of the bright area 33 (that is, four color charts in the chart 35) and the dark area 34 (that is, four color charts in the chart 36). An approximate curve is generated independently. The method of deriving the approximate curve is the same as in the case of FIG. In FIG. 9B, the curve 13 is an approximate curve in the dark region 34, and the curve 14 shows an example of the approximate curve in the bright region 33. A plurality of profiles are generated from the plurality of approximate curves derived in this way. In generating the profile, the test image pixel value and the brightness score may be converted into, for example, a linear form as an input value and an output value, respectively.

  FIG. 10 is a diagram for explaining a generated gradation conversion profile. FIG. 10A shows a profile generated with respect to FIG. 9A described above. On the other hand, FIG. 10B is a profile generated for FIG. 9B. As the profile format, a known format in which the correspondence between the characteristics of the input image data in the predetermined color space and the characteristics of the corrected image data in the predetermined color space is recorded can be used. For example, an ICC profile proposed by ICC (International Color Consortium) can be used. As the color space of the image data before correction, a standard color space that does not depend on the color reproduction range and color reproduction characteristics of the image input / output device, such as an XYZ color system, is suitable. On the other hand, as the color space of the corrected image data, for example, sRGB or AdobeRGB, which are color spaces generally used in display devices, are suitable.

  A curve 21 in FIG. 10A shows the characteristics of a single profile (A0001) used for gradation conversion for an input image. On the other hand, a curve 22 in FIG. 10B is a characteristic of the profile (A0003) used for gradation conversion for the dark area of the input image, and a curve 23 is a profile (A0002) used for gradation conversion for the light area of the input image. Each characteristic is shown. As described above, when inversion occurs between the pixel value and the brightness score in the brightness score designation test result, a different profile is generated for each illuminance area. This makes it possible to provide a corrected image that matches the brightness perceived when viewing the scene.

  The profiles stored in the profile DB 203 are classified based on, for example, the presence or absence of reversal in the result of the brightness point designation test by the above-mentioned pre-profile creator. Further, for each classification, a plurality of brightness score results for each color chart may be averaged, and a gradation conversion curve with the average value as the brightness score of each color chart may be derived to generate a profile for each classification.

  FIG. 11 is a diagram illustrating an example of profile selection by the profile selection unit 202. The profiles A0001 to A0003 described above are stored in the profile DB 106 together with the test image 31 used when each profile is generated.

  As described above, the profile is selected based on the presence or absence of reversal. Here, when there is no reverse rotation, the profile A0001 corresponding to the curve 21 in FIG. 10A is selected. On the other hand, when there is reversal, the profile A0003 corresponding to the curve 22 in FIG. 10B and the profile A0002 corresponding to the curve 23 are selected.

  As described above, according to the first embodiment, the presence / absence of reversal of the brightness perception and the pixel value for the object under the bright region and the dark region is derived for each user. Then, the gradation conversion profile to be used is changed based on the presence or absence of inversion. As a result, it is possible to provide a tone-corrected image that is less uncomfortable for the user, particularly during tone conversion involving dynamic range conversion. Further, it can also be used for gradation conversion at the time of editing a photographed video in video production or at the time of calibration of a display device.

(Second Embodiment)
In the second embodiment, a mode in which a brightness point designation test is performed on a user and a profile corresponding to the brightness perception of the user is directly generated from the test result will be described. Specifically, in S2, the brightness point designation test (FIG. 8) is performed on the user instead of the brightness order designation test (FIG. 4), and the profile is dynamically generated as in the first embodiment. Different. Note that the configuration (FIG. 1) and operation (FIG. 2) of the apparatus are the same as those in the first embodiment, and a detailed description thereof will be omitted.

<Details of brightness result specification (S2)>
As described above, in the second embodiment, the brightness point designation test shown in FIG. 8 is performed on the user. The details of the point designation test are the same as those in the first embodiment, and a description thereof will be omitted. As in the case of the point designation test in the first embodiment, the brightness point designation test result as shown in FIG. 9 is obtained.

  That is, there is no inversion between the order of the pixel values of each color chart and the order of the brightness points as shown in FIG. 9A, and the order of the pixel values of each color chart as shown in FIG. 9B. One of the results is obtained when there is a reversal between the order of brightness and the number of brightness points. Depending on the difference in the result of the brightness point designation test, the profile generation process in the profile changing unit 105 branches.

<Details of profile change (S3)>
FIG. 12 is a diagram illustrating an example of the configuration of the profile changing unit 105 according to the second embodiment.

  The brightness order determination unit 201 determines whether or not there is a reversal between the brightness order of each color chart designated by the brightness designation section 104 and the order of the pixel values of each color chart. The profile generation unit 302 generates a profile based on the brightness point designation test result shown in FIG.

  FIG. 13 is a flowchart for explaining the processing of the profile changing unit 105 in the second embodiment.

  In step S <b> 41, the brightness order determination unit 201 receives a brightness point designation test result from the brightness designation unit 104. That is, the brightness score for each color chart input in the dialog box 52 is accepted.

  In step S <b> 42, the brightness order determination unit 201 determines whether or not there is an inversion between the brightness score and the rank order of the pixel values of each color chart in the received brightness score designation test result. Then, information on “presence / absence of reverse rotation” and “test image used” is supplied to the profile generation unit 302. If there is no reverse rotation, the process proceeds to step S43, and if there is a reverse rotation, the process proceeds to step S44.

  In step S43, the profile generation unit 302 generates a single profile shown in FIG. On the other hand, in step S44, the profile generation unit 302 generates a plurality of profiles. Note that the profile generation method is the same as that in the first embodiment (FIGS. 8 to 10), and a description thereof will be omitted.

  In step S <b> 45, the profile generation unit 302 supplies the generated profile to the image correction unit 106. Note that the correction processing in the image correction unit 106 is the same as that in the first embodiment, and thus the description thereof is omitted.

  As described above, according to the second embodiment, the presence / absence of the reversal of the brightness perception and the pixel value for the object under the bright region and the dark region is derived for each user. Then, a gradation conversion profile to be used is generated based on the presence or absence of inversion. As a result, it is possible to provide a gradation-corrected image that is less uncomfortable for the user.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  DESCRIPTION OF SYMBOLS 101 Image processing apparatus; 102 Test image data; 103 Image data input part; 104 Brightness designation part; 105 Profile change part; 106 Image correction part; 107 Image data output part;

Claims (10)

An image processing apparatus that performs gradation conversion processing on an input image,
A test image including a plurality of color chart images photographed under a first illuminance and a plurality of color chart images photographed under a second illuminance darker than the first illuminance is displayed on a display unit. Display control means for
Receiving means for receiving an input of a perceptual value related to brightness for each color chart in the test image displayed on the display unit;
Determining means for determining a profile to be used for the gradation conversion processing based on a perceptual value for each color chart received by the receiving means;
Correction means for performing a gradation conversion process on the input image using the profile determined by the determination means to generate a corrected image;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the correction unit performs the gradation conversion processing such that a dynamic range of the corrected image is narrower than a dynamic range of the input image. If the order of the perceived values for the color charts received by the receiving means does not match the order of the pixel values in the color charts of the test image, the determining means corresponds to a different brightness area. The image processing apparatus according to claim 1, wherein a plurality of profiles are determined. The correction unit divides the input image into a plurality of brightness regions corresponding to the plurality of profiles when the plurality of profiles are determined by the determination unit, and profiles corresponding to the plurality of brightness regions. The image processing apparatus according to claim 3, wherein the correction image is generated by performing a gradation conversion process using the image. The image processing apparatus according to claim 1, wherein the reception unit receives an input of the order of the perceptual values. A storage unit for storing a plurality of profiles;
6. The image processing according to claim 1, wherein the determination unit selects a profile to be used for the gradation conversion process from a plurality of profiles stored in the storage unit. apparatus. The determination unit generates a profile used for the gradation conversion processing based on a perception value for each color chart received by the reception means and a pixel value in each color chart of the test image. 6. The image processing apparatus according to any one of items 1 to 5. The determining means generates a single profile when the order of the perceived value magnitude for each color chart received by the accepting means matches the order of the pixel value magnitude in each color chart of the test image, The image processing apparatus according to claim 7, wherein if there is no match, a plurality of profiles corresponding to different brightness regions are generated. A control method of an image processing apparatus that performs gradation conversion processing on an input image,
A test image including a plurality of color chart images photographed under a first illuminance and a plurality of color chart images photographed under a second illuminance darker than the first illuminance is displayed on a display unit. Display process to
An accepting step of accepting an input of a perceptual value relating to brightness for each color chart in the test image displayed on the display unit;
A determination step of determining a profile to be used for the gradation conversion processing based on a perceptual value for each color chart received in the reception step;
A correction step of generating a correction image by performing the gradation conversion processing on the input image using the profile determined in the determination step;
A control method for an image processing apparatus, comprising:   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 8.

Images