WO2018159162A1 - 画像処理装置およびその制御方法及びプログラム - Google Patents

画像処理装置およびその制御方法及びプログラム Download PDF

Info

Publication number
WO2018159162A1
WO2018159162A1 PCT/JP2018/002127 JP2018002127W WO2018159162A1 WO 2018159162 A1 WO2018159162 A1 WO 2018159162A1 JP 2018002127 W JP2018002127 W JP 2018002127W WO 2018159162 A1 WO2018159162 A1 WO 2018159162A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image data
luminance
correction
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/002127
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
敬行 神野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Publication of WO2018159162A1 publication Critical patent/WO2018159162A1/ja
Priority to US16/548,039 priority Critical patent/US11328679B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/34—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
    • G09G3/3406—Control of illumination source
    • G09G3/3413—Details of control of colour illumination sources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/001—Texturing; Colouring; Generation of texture or colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/10—Intensity circuits
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40—Picture signal circuits
    • H04N1/407—Control or modification of tonal gradation or of extreme levels, e.g. background level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0613—The adjustment depending on the type of the information to be displayed
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0673—Adjustment of display parameters for control of gamma adjustment, e.g. selecting another gamma curve
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/08—Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/14—Detecting light within display terminals, e.g. using a single or a plurality of photosensors
    • G09G2360/144—Detecting light within display terminals, e.g. using a single or a plurality of photosensors the light being ambient light
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/16—Calculation or use of calculated indices related to luminance levels in display data

Definitions

  • the present invention relates to an image processing apparatus, a control method thereof, and a program.
  • the luminance range of a subject has been adjusted so far by combining multiple images taken under different exposure conditions or using an imaging device that expands the luminance range (dynamic range) that can be captured.
  • an imaging device that expands the luminance range (dynamic range) that can be captured.
  • HDR high dynamic range
  • high-intensity subjects such as light reflected on high-gloss objects such as automobile headlights, the sun, or metals, are included in the shooting target, so It becomes possible to widen the range of expression.
  • Patent Document 1 discloses a technique related to a flare correction circuit generated in a display device. This is an edge emphasizing technique in which flare in the display is a problem in which the edge portion is blurred due to the MTF characteristic of a projection lens of a display device such as a projector. Although the periphery of the high-luminance subject becomes bright due to the blur at the edge portion, the cause is on the display device side, but the problem is similar in that the observer perceives flare. In particular, when the surroundings are bright, human visual sensitivity is reduced and the edge enhancement effect is diminished, so the flare correction gain and band are automatically adjusted according to the surrounding brightness.
  • Patent Document 1 discloses a technique for correcting flare of a display device in accordance with ambient brightness, but discloses a technique for correcting artifacts such as flare based on the luminance of a displayed image itself. I don't mean.
  • the present invention has been made in view of the above problems, and is intended to provide a technique for correcting image artifacts related to luminance and color derived from a high luminance region to be displayed based on the luminance of a display device. is there.
  • an image processing apparatus of the present invention has the following configuration. That is, An image processing device for correcting image data to be displayed on a display device, Obtaining means for obtaining information corresponding to a luminance range displayable by the display device; Derivation means for deriving a correction amount for correcting artifacts of image data to be displayed based on the maximum luminance indicated by the information acquired by the acquisition means; Correction means for correcting the image data based on the derived correction amount and generating display image data.
  • the present invention it is possible to correct image artifacts related to luminance and color derived from a high luminance area to be displayed in consideration of luminance of the display device. As a result, it is possible to suppress the uncomfortable feeling of artifacts when observing a display device capable of high brightness display.
  • FIG. 1 is a schematic block configuration diagram of an image processing apparatus according to a first embodiment.
  • 5 is a flowchart illustrating a processing procedure of the image processing apparatus according to the first embodiment.
  • the figure which shows the content of the correction table of embodiment The figure which shows the content of the correction table of embodiment.
  • the figure for demonstrating the correction method of the gradation conversion process in the artifact correction execution part of 1st Embodiment The figure which shows the content of the correction table in 2nd Embodiment.
  • the figure which shows the content of the correction table in 2nd Embodiment The figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an artifact.
  • the figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an artifact The figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an artifact.
  • the figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an artifact The figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an art
  • the figure for demonstrating the detection method of the artifact in 2nd Embodiment, and the addition method of an artifact The figure which shows the execution example of the artifact correction
  • FIG. 10 is a schematic configuration diagram of an image processing apparatus according to a third embodiment.
  • FIG. 1 is a conceptual diagram showing a schematic configuration of the image processing apparatus according to the present embodiment.
  • the apparatus includes an image acquisition unit 101, an artifact correction unit 102, an image display unit 103, and a display luminance acquisition unit 104, as illustrated.
  • the image acquisition unit 101 is an image editing application or an image display application in an information processing apparatus such as a personal computer (PC), and any type may be used as long as it has a function of acquiring an image to be displayed.
  • a typical example of a generation source of an image to be displayed is an imaging device such as a digital camera.
  • the artifact correction unit 102 derives a correction amount based on the display luminance information acquired by the display luminance acquisition unit 104 described later, and performs an artifact correction process on the image acquired by the image acquisition unit 101 based on the derived correction amount.
  • the display luminance acquisition unit 104 acquires display luminance information used for artifact correction of the artifact correction unit 102.
  • the image display unit 103 includes an image display device such as a display or a projector, and receives and displays the display image data on which the artifact correction by the artifact correction unit 102 has been performed.
  • the display luminance information is the maximum luminance that can be output by the image display device of the image display unit 103, for example.
  • the display luminance information is the maximum display luminance corresponding to the display target image determined from the maximum luminance that can be output by the image display device of the image display unit 103 and the maximum pixel value of the image acquired by the image acquisition unit 101. It is.
  • a table or a conversion formula indicating the correspondence between the pixel value and the display luminance as shown in the graph of FIG. 3 is used. This correspondence is assumed to be stored in the memory of the image display device of the image display unit 103.
  • a database having a correspondence relationship between the model name of the display device and the notation luminance value is prepared in advance and actually used.
  • a display device When a display device is designated, it may be obtained by referring to the table. Therefore, a luminance slightly lower than the maximum luminance may be used instead of the strict maximum luminance. That is, it is only necessary to acquire luminance information indicating luminance that can be classified as high luminance and close to the maximum luminance in the luminance range that can be output by the image display device, and a range that includes luminance close to the maximum luminance may be acquired.
  • the maximum luminance includes this output luminance information.
  • the maximum pixel value of the low frequency component of the image of the image acquisition unit 101 instead of simply adopting the maximum pixel value of the image acquired by the image acquisition unit 101. That is, the maximum display luminance corresponding to the display target image determined from the maximum luminance that can be output by the image display device of the image display unit 103 and the maximum pixel value of the low frequency component of the image acquired by the image acquisition unit 101 is displayed. Let it be luminance information.
  • the display luminance acquisition unit 104 includes a conversion unit that converts the image acquired by the image acquisition unit 101 into a low-frequency image. This conversion unit may be filter processing or resolution conversion. Then, the display luminance acquisition unit 104 obtains the maximum pixel value (maximum luminance value) of the low frequency component.
  • FIG. 4A shows a part of an image acquired by the image acquisition unit 101, and the value of each pixel indicates a pixel value.
  • a value having a high correlation with the brightness of the image such as a Y value of an image converted from an RGB image to a Yuv image, is used.
  • FIG. 4B shows pixel values of an image that has been subjected to resolution conversion processing using the bicubic method, which is a known resolution conversion method, with respect to FIG. 4A.
  • the resolution conversion method is not limited to this, and a known method such as a bilinear method, a Lanczos method, or an average pixel method can be used.
  • the thick frame pixel in FIG. 4A is used.
  • the display luminance information is obtained from the low frequency component of the image acquired by the image acquisition unit 101
  • the thick frame in FIG. A pixel value is obtained from each pixel. Display luminance information is obtained from these pixel values and the correspondence shown in FIG.
  • the display luminance acquired from the image acquisition unit 101 can be used as display luminance information.
  • the display luminance information is not necessarily limited to luminance, and may be different information such as CIE brightness as long as it is information related to brightness.
  • FIG. 2 is a block diagram of the artifact correction unit 102 in the embodiment.
  • the correction presence / absence determination unit 201 determines whether to perform artifact correction based on the display luminance information acquired by the display luminance acquisition unit 104. Specifically, the correction presence / absence determination unit 201 compares the luminance value indicated by the display luminance information with a preset threshold value, and determines that correction is performed when the luminance value is equal to or greater than the threshold value, and is less than the threshold value. In this case, it is determined that there is no correction.
  • the threshold value can be set and changed by the user via the operation unit, and it is desirable to store the threshold value in a non-volatile memory for easy holding unless there is a change instruction.
  • the correction presence / absence determination unit 201 determines not to perform correction
  • the image data acquired by the image acquisition unit 101 is output to the image display unit 103 without performing any processing in subsequent processing blocks. And display.
  • the correction amount calculation unit 203 receives the determination result and calculates a correction amount for correcting the artifact. Although a calculation method of the correction amount will be described later, the correction amount is obtained with reference to the correction table 202 based on the display luminance information acquired by the display luminance acquisition unit 104.
  • the correction amount calculation of the correction amount calculation unit 203 it is possible to calculate the correction amount by further using not only the display luminance information but also the environmental light intensity information acquired by the environmental light acquisition unit 204.
  • This ambient light intensity may be acquired by an illuminance sensor mounted on a display device such as a display, or may be a value measured around the display device by another illuminometer. Alternatively, the user may set the value via an operation unit (not shown).
  • the artifact detection unit 205 detects the artifact of the image acquired by the image acquisition unit 101 when it is determined to be corrected by the correction presence / absence determination unit 201 (details will be described later).
  • the artifact correction execution unit 206 executes artifact correction based on the correction amount calculated by the correction amount calculation unit 203 and the artifact detection result detected by the artifact detection unit 205. Note that the artifact detection unit 205 is not necessarily an essential component, although it depends on the content of the correction processing of the artifact correction execution unit 206.
  • FIG. 5 is a flowchart showing the overall processing flow in the image processing apparatus according to this embodiment.
  • the image acquisition unit 101 acquires an image to be displayed on the image display device of the image display unit 103 (S501).
  • the display luminance acquisition unit 104 acquires display luminance information for display on the image display device of the image display unit 103 (S502).
  • the artifact correction unit 102 corrects the artifacts related to luminance and color derived from the high luminance area of the image based on the display luminance information acquired in S502 for the image acquired in S501.
  • FIG. 5 is also a flowchart showing the flow of the artifact correction unit 102.
  • the artifact correction unit 102 causes the correction presence / absence determination unit 201 to compare the value of the display luminance information acquired by the display luminance acquisition unit 104 with a preset threshold value (S601). If the artifact correction unit 102 determines that the value of the display luminance information is greater than or equal to this threshold, the process proceeds to S602. If the artifact correction unit 102 determines that the value of the display luminance information is less than the threshold value, the artifact correction unit 102 determines not to perform the artifact correction, and ends this process. Next, the artifact correction unit 102 causes the ambient light acquisition unit 204 to acquire ambient light intensity such as illuminance of the image display device or its surroundings (S602).
  • the artifact correction unit 102 controls the correction amount calculation unit 203 to determine the correction amount in the artifact correction process with reference to the correction table 202 based on the value of the display luminance information acquired by the display luminance acquisition unit 104 ( S603).
  • the artifact correction unit 102 controls the artifact detection unit 205 to detect the artifact of the image acquired by the image acquisition unit 101 (S604). Finally, the artifact correction unit 102 controls the artifact correction execution unit 206 to perform artifact correction on the image acquired by the image acquisition unit 101 based on the correction amount determined in S603 and the artifact detected in S604. This is executed (S605). As described above, depending on the content of the artifact correction process, the detection of the artifact is not necessarily required. In the configuration without the artifact detection unit 205, the artifact correction execution unit 206 performs the artifact correction from only the correction amount calculated in S603 in S605. Will be executed.
  • FIG. 7A is a graph showing the contents of the correction table 202
  • FIG. 7B shows a specific example of a lookup table (LUT).
  • the graph in FIG. 7A shows the relationship between the display brightness acquired by the display brightness acquisition unit 104 and the correction amount. It can be seen that the correction amount increases as the value of the display luminance information increases.
  • FIG. 7B is a specific LUT of the display luminance value and the correction amount having the relationship shown in FIG. 7A.
  • the correction table 202 holds data in the LUT format as shown in FIG. 7B.
  • the correction amount of the display luminance in the LUT having the smaller difference from the acquired luminance value is set. Decide what to use.
  • the correction amount may be obtained by interpolation using a known interpolation technique such as linear interpolation.
  • the correction amount may be obtained by extrapolation by a known extrapolation technique such as linear extrapolation.
  • the correction amount is set to 100% when the display luminance is 10,000 nits.
  • the correction amount is not limited to this.
  • the correction amount may be set to 100% when the display luminance is 1000 nits.
  • the relationship between the display luminance and the correction amount may be held as a “function” in addition to the LUT format.
  • the correction amount calculation unit 203 calculates a correction amount based on the display luminance acquired by the display luminance acquisition unit 104 and the environmental light intensity acquired by the environmental light acquisition unit 204.
  • FIG. 8A shows an example of a correction LUT in which the correction amount is increased as the ambient light intensity value is smaller.
  • FIG. 8B shows an example of a correction LUT in which the correction amount is increased as the environmental light intensity value is larger.
  • FIG. 9A shows an example of an image acquired by the image acquisition unit 101.
  • FIG. 9B shows a binary image obtained by performing binarization processing based on a predetermined threshold set in advance for the image of FIG. 9A.
  • a white area indicates a high brightness area
  • a black area indicates a non-high brightness area.
  • the pixel value of the non-high brightness area of the binary image is considered to be zero.
  • the high luminance area of the binary image is set as the average luminance value of the actual image in the area.
  • This binary image represents the determination result of the correction presence / absence determination unit 201.
  • the artifact detection unit 205 calculates the average value of each of R, G, and B from the actual image shown in FIG. 9A corresponding to the region shown in white in FIG. Convert to Y, u, v of the color system. Also, the barycentric coordinates of the area are acquired.
  • information on brightness such as luminance, information on color, and information on position on the image are acquired.
  • a plurality of high-luminance regions may be extracted, but it is preferable to set an upper limit number in advance and determine it from the size and area of the average pixel value.
  • FIG. 10A is an artifact component image detected from the image of FIG. 9A.
  • FIG. 10B shows the pixel distribution crossed by the broken line shown in FIG. 10A in the binary image of FIG. 9B (the binary image in which the high luminance area is the actual average luminance value and the non-high luminance area is zero).
  • the horizontal axis indicates the position on the line, and the vertical direction is the luminance value.
  • FIG. 10C is a graph showing the relationship between the position of the image obtained by blurring FIG. 9B with a predetermined filter and the pixel value, and shows the pixel value on the line indicated by the dotted line in FIG. 10A. Note that a Gaussian filter as shown in FIG.
  • FIG. 11 can be used as a filter for blurring an image from which a high luminance region is extracted, but another filter such as a media filter may be used as long as it is a blur filter.
  • FIG. 11 shows an example of a 5 ⁇ 5 size filter, the filter size and coefficients are not limited to this example.
  • the artifact detection unit 205 stores a plurality of filters, and applies a filter of an appropriate size depending on the image.
  • FIG. 10D is a graph showing the relationship between the position of the image obtained by subtracting the image of FIG. 9B from the image blurred by the predetermined filter and the pixel value, and showing the pixel value on the line indicated by the dotted line in FIG. 10A. Yes. That is, the pixel value profile in FIG.
  • 10A corresponds to the profile in the dotted line of the artifact component shown in FIG. If the pixel value is smaller than 0 when the image in FIG. 9B is subtracted from the image obtained by blurring FIG. 9B with a predetermined filter, the value is clipped to 0.
  • FIG. 12A is an image acquired by the image acquisition unit 101 corresponding to FIG. 9A.
  • FIG. 12B is a corrected artifact component image in which the correction amount calculated by the correction amount calculation unit 203 is applied to the image indicating the artifact component of FIG. 10B generated (detected) by the artifact detection unit 205.
  • the correction amount ⁇ is determined to be 14.6% from the correction table of FIG. 7B.
  • FIG. 12C is an image obtained as a result of subtracting the corrected artifact component image of FIG. 12B from the image of FIG. 12A as the artifact correction processing. Again, if the pixel value is smaller than 0 during the subtraction process, the value is clipped to 0. As shown in FIG. 12B, based on the position and color of the high luminance component of the input image, different amounts of correction are performed depending on the position on the image.
  • one pixel of the image of FIG. 12A has an RGB component
  • one pixel of the corrected artifact component image of FIG. 12B has only one luminance component. Therefore, the artifact correction execution unit temporarily converts the image in FIG. 12A into the Yuv color space, and subtracts the luminance component in FIG. 12B from the luminance Y component to obtain a corrected luminance value Y ′.
  • the artifact correction execution unit reconverts Y ′, u, v into an RGB color space image and outputs the image to the image display unit 103. Note that the image display unit 103 does not need the last reconversion when it can display a Yuv format image.
  • the digital gain based on the correction amount calculated by the correction amount calculation unit 203 is applied to the artifact component in FIG. 10A.
  • the correction amount can also be controlled by generating a plurality of artifact component images using a plurality of filters having different sizes and coefficients, and selecting them based on the correction amount calculated by the correction amount calculation unit 203.
  • FIG. 13A and 13B show Laplacian filters having different intensities.
  • FIG. 13A shows an example of a relatively weak filter
  • FIG. 13B shows an example of a relatively strong filter.
  • only two filters are shown as an example, but it is preferable to store more filters in the artifact correction execution unit 206 so as to cover the correction amount calculated by the correction amount calculation unit 203. .
  • the artifact correction execution unit 206 controls the correction amount by performing filter processing using a filter having a stronger intensity as the correction amount is larger.
  • FIG. 14A is a graph of gamma characteristics showing the relationship of the output pixel value to the input pixel value.
  • the artifact correction execution unit 206 controls the correction amount by selecting a gamma characteristic in which the pixel value decreases as the correction amount increases and performs gradation conversion processing. To do. That is, the gamma characteristic 1402 shows an example of a gamma characteristic when the correction amount is relatively small, and the gamma characteristic 1403 shows an example of a gamma characteristic when the correction amount is relatively large.
  • FIG. 14B is a graph showing the relationship between the position and the pixel value in a line passing through the center of the sun, which is the high luminance region in FIG. 9A.
  • Reference numeral 1404 is a pixel value when correction is not performed.
  • Reference numeral 1405 indicates a pixel value when converted by the gamma characteristic of the gamma characteristic 1402, and reference numeral 1406 indicates a pixel value when converted by the gamma characteristic of the gamma characteristic 1403. It can be seen that the pixel value curve 1406 to which the gamma characteristic 1403 when the correction amount calculated by the correction amount calculation unit 203 is large is corrected more than the pixel value curve 1405.
  • correction processing may be performed on the RGB values or may be performed on the converted Yuv.
  • the present embodiment it is possible to perform correction so that the higher the luminance of the image displayed on the image display device, the smaller the luminance and color artifact components derived from the high luminance region of the image. As a result, it is possible to suppress a sense of discomfort derived from artifacts when observing a display device capable of high brightness display.
  • FIG. 15A is a correction curve indicated by the correction table 202
  • FIG. 15B is a specific look-up table (LUT).
  • the graph in FIG. 15A shows the relationship between the display brightness acquired by the display brightness acquisition unit 104 and the correction amount. It can be seen that the correction amount increases as the value of the display luminance information decreases.
  • FIG. 15B is an LUT describing the relationship shown in FIG. 15A.
  • the correction table 202 is stored in the LUT format as shown in FIG. 15B.
  • the correction amount corresponding to the closest display luminance is referred to.
  • the correction amount may be obtained by interpolation using a known interpolation technique such as linear interpolation.
  • the relationship between the display luminance and the correction amount may be held as a function.
  • correction may be performed based on the ambient light intensity acquired by the ambient light acquisition unit 204, and the correction table 202 holds LUTs for a plurality of ambient light intensities. It is preferable.
  • Artifact Correction Processing A method of detecting an artifact by the artifact detection unit 205 and a method of adding the artifact correction execution unit 206 in the second embodiment will be described with reference to FIGS. 16A to 16E.
  • FIG. 16A is an artifact component detected from the image of FIG. 9A as in FIG. 10A.
  • FIG. 16B is a graph showing the relationship between the position of the image and the pixel value corresponding to FIG. 9A, as in FIG. 10B.
  • FIG. 16C is a graph showing the relationship between the position of the image obtained by blurring FIG. 9B with a predetermined filter and the pixel value, as in FIG. 10C.
  • the dotted lines in FIGS. 16D and 16E are graphs showing the relationship between the position of the image obtained by subtracting the image of FIG. 9B from the image obtained by blurring the image of FIG. 9B) with a predetermined filter and the pixel values, and showing the artifact component of FIG. Yes.
  • FIG. 16D and 16E indicate the reference values of the artifact components added by the artifact correction execution unit 206.
  • a solid line in FIG. 16D indicates a pixel value obtained by applying a digital gain based on the correction amount to the pixel value indicated by the dotted line.
  • the digital gain in FIG. 16D is a gain having such a characteristic that the value increases as the distance from the high luminance region increases, and the luminance value to be added depends on the correction amount.
  • FIG. 16E shows a pixel value obtained by applying a digital gain based on the correction amount to the pixel value indicated by the dotted line, as in FIG. 16D.
  • 16E is a gain having such a characteristic that the value becomes higher as the distance from the high luminance region is closer, and the luminance value to be added corresponds to the correction amount. Note that the image with gain is clipped at the maximum pixel value in the high luminance region.
  • either of the solid lines in FIGS. 16D and 16E may be used. This selection is made to be selected by the user.
  • FIGS. 17A to 17D are diagrams illustrating an execution example of the artifact correction in the artifact correction execution unit 206 according to the second embodiment.
  • 17A is an image acquired by the image acquisition unit 101 corresponding to FIG. 9A
  • FIG. 17B is an artifact component detected by the artifact detection unit 205 corresponding to FIG. 16A.
  • FIG. 17C shows an artifact component to be added.
  • This artifact component is an image obtained by applying a digital gain based on the correction amount calculated by the correction amount calculation unit 203 to the reference value of the artifact component to be added, which is indicated by the solid line in FIG. 16D or FIG. 16E.
  • the artifact correction execution unit 206 first subtracts (removes) the artifact component shown in FIG. 17B from the original image shown in FIG. 17A. If the pixel value after subtraction is less than 0, the value is clipped to 0. Next, the artifact correction execution unit 206 adds the artifact component of FIG. 17C to the image after subtraction.
  • FIG. 17D shows an image of this processing result, and it can be seen that the artifact component is increased.
  • FIG. 18A is a gamma characteristic graph showing the relationship between the input pixel value and the output pixel value.
  • Reference numerals 1802 and 1803 have gamma characteristics that increase the output pixel value, compared to reference numeral 1801 when gradation conversion is not performed.
  • the artifact correction execution unit 206 controls the correction amount by selecting a gamma characteristic in which the pixel value increases as the correction amount increases and performs gradation conversion processing. To do.
  • reference numeral 1802 indicates an example of gamma characteristics when the correction amount is relatively small
  • reference numeral 1803 indicates an example of the gamma characteristic when the correction amount is relatively large
  • FIG. 18B is a graph showing the relationship between the position and the pixel value in a line passing through the center of the sun, which is the high luminance region in FIG. 9A.
  • Reference numeral 1804 denotes a curve of pixel values when correction is not performed.
  • Reference numeral 1805 indicates a curve of the pixel value when converted by the gamma characteristic 1802, and reference numeral 1806 indicates a curve of the pixel value when converted by the gamma characteristic 1803.
  • the lower the luminance of the image displayed on the image display device the larger the artifact component relating to the luminance and color derived from the high luminance region of the image can be corrected. .
  • FIG. 19 is a conceptual diagram showing a schematic configuration of the image processing apparatus according to the third embodiment.
  • the image acquisition unit 1901 acquires an image to be displayed.
  • the generation source of the acquired image is not particularly limited.
  • the first artifact correction unit 1902 is a process similar to the artifact correction process described in the first embodiment for the image acquired by the image acquisition unit 1901 based on the display luminance information acquired by the display luminance acquisition unit 1903 described later. Apply.
  • the display luminance acquisition unit 1903 acquires display luminance information used for artifact correction of the first artifact correction unit 1902.
  • the second artifact correction unit 1904 is an image acquired by the image acquisition unit 1901 or an image subjected to correction processing by the first artifact correction unit 1902 based on a correction amount adjusted by a user by a correction amount adjustment unit 1905 described later. Perform artifact correction for.
  • the correction amount adjustment unit 1905 has a user interface (UI) that allows the user to adjust the correction amount of the artifact, and determines the correction amount of the second artifact correction unit 1904.
  • the image display unit 1906 includes an image display device.
  • the image display unit 1905 is based on the image acquired by the image acquisition unit 1901, the image in which the artifact is automatically corrected based on the display luminance by the first artifact correction unit 1902, and the adjustment by the user in the second artifact correction unit 1904. Display images with corrected artifacts for comparison.
  • the image display unit 1906 may not display all of these images at the same time, but may display two of these images at the same time, or may display the images while switching them one by one.
  • the image selection unit 1907 includes a UI that allows the user to select an image using the image display of the image display unit 1906.
  • FIG. 20 is a UI (User Interface) for the user to adjust the correction amount for artifact correction, which is included in the correction amount adjustment unit 1905.
  • This UI has a toggle button for selecting an image as a reference for correction from an image acquired by the image acquisition unit 1901 and an image subjected to correction processing by the first artifact correction unit 1902. Is preferred.
  • the illustrated radio buttons “no correction” and “automatic correction” are examples of this alternative.
  • the image used as a reference for correction is not necessarily selected from these images, and either one of the reference images may be used.
  • this UI has a function of displaying an image corrected based on the correction amount adjusted by the user. The user determines the correction amount while confirming the correction result. In the figure, it is shown that the user designates the amount of correction with a slider bar.
  • FIG. 21 shows a UI for the user to select an image, which is included in the image selection unit 1907.
  • This UI is selected from the image acquired by the image acquisition unit 1901, the image corrected by the first artifact correction unit 1902, and the image corrected by the second artifact correction unit 1904 based on the adjustment instruction from the user. It has a toggle button that can be selected at one time.
  • the images to be selected need not necessarily be all of these images, and may be configured to display two of these images.
  • FIG. 22A and 22B show image display examples on the image display unit 1906.
  • FIG. FIG. 22A shows a method for displaying images to be compared side by side
  • FIG. 22B shows an example in which different images are displayed on the left and right of the divided areas.
  • the dividing boundary position can be moved to the left and right by using a mouse pointer as shown in the drawing, a keyboard not shown in the drawing, or the like.
  • the display position in FIG. 22A and the division direction in FIG. 22B are not limited to this.
  • the image to be displayed is based on the image acquired by the image acquisition unit 1901, the image subjected to the correction processing by the first artifact correction unit 1902, and the correction amount adjusted by the user by the second artifact correction unit 1904. You may display the corrected image simultaneously.
  • the user can adjust the correction amount of the artifacts related to the luminance and color derived from the high luminance region of the image due to the difference in luminance of the display device.
  • the present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
  • a circuit for example, ASIC

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Transforming Electric Information Into Light Information (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)
  • Facsimile Image Signal Circuits (AREA)
PCT/JP2018/002127 2017-03-02 2018-01-24 画像処理装置およびその制御方法及びプログラム Ceased WO2018159162A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/548,039 US11328679B2 (en) 2017-03-02 2019-08-22 Image processing apparatus, control method thereof, and non-transitory computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-039772 2017-03-02
JP2017039772A JP2018148318A (ja) 2017-03-02 2017-03-02 画像処理装置およびその制御方法及びプログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/548,039 Continuation US11328679B2 (en) 2017-03-02 2019-08-22 Image processing apparatus, control method thereof, and non-transitory computer-readable storage medium

Publications (1)

Publication Number Publication Date
WO2018159162A1 true WO2018159162A1 (ja) 2018-09-07

Family

ID=63371100

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/002127 Ceased WO2018159162A1 (ja) 2017-03-02 2018-01-24 画像処理装置およびその制御方法及びプログラム

Country Status (3)

Country Link
US (1) US11328679B2 (enExample)
JP (2) JP2018148318A (enExample)
WO (1) WO2018159162A1 (enExample)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102579457B1 (ko) * 2017-02-24 2023-09-14 삼성전자주식회사 이미지 보정 방법 및 장치
JP2018148318A (ja) * 2017-03-02 2018-09-20 キヤノン株式会社 画像処理装置およびその制御方法及びプログラム
CA3114112A1 (en) * 2018-09-26 2020-04-02 Dolby Laboratories Licensing Corporation Projector light source dimming using metadata from future frames
WO2020203237A1 (ja) * 2019-03-29 2020-10-08 ソニー株式会社 画像処理装置、および画像処理方法、並びにプログラム
JP7484421B2 (ja) * 2020-05-26 2024-05-16 コニカミノルタ株式会社 色パラメーター生成装置、実行装置および色パラメーター生成プログラム
CN112381836B (zh) * 2020-11-12 2023-03-31 贝壳技术有限公司 图像处理方法和装置、计算机可读存储介质、电子设备
WO2022119372A1 (ko) 2020-12-03 2022-06-09 삼성전자주식회사 영상 처리를 수행하는 전자 장치 및 그 동작 방법
CN115880164A (zh) * 2021-09-28 2023-03-31 西安诺瓦星云科技股份有限公司 图像的校正方法、装置和存储介质
TWI807868B (zh) 2022-06-17 2023-07-01 星彩顯示股份有限公司 具環境適應性之Gamma自動調校系統及其方法
US20240078946A1 (en) * 2022-09-02 2024-03-07 Apple Inc. Display Pipeline Compensation for a Proximity Sensor Behind Display Panel

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001203933A (ja) * 2000-01-19 2001-07-27 Hitachi Ltd 撮像装置
JP2002077591A (ja) * 2000-09-05 2002-03-15 Minolta Co Ltd 画像処理装置および撮像装置
JP2005295513A (ja) * 2004-03-05 2005-10-20 Matsushita Electric Ind Co Ltd 画像信号処理方法、画像信号処理装置および画像表示装置
JP2012085083A (ja) * 2010-10-12 2012-04-26 Nikon Systems Inc 画像処理装置、撮像装置および画像処理プログラム
JP2012134630A (ja) * 2010-12-20 2012-07-12 Nikon Corp 画像処理方法、画像処理プログラム、画像処理装置、及び撮像装置
WO2017022513A1 (ja) * 2015-07-31 2017-02-09 ソニー株式会社 映像信号処理装置および映像信号処理方法、ならびに表示装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4574057B2 (ja) * 2000-05-08 2010-11-04 キヤノン株式会社 表示装置
DE602005021186D1 (de) * 2004-03-05 2010-06-24 Panasonic Corp Bildsignalverarbeitungsverfahren, bildsignalverarbeitungsvorrichtung und bildanzeigevorrichtung
US8320641B2 (en) * 2004-10-28 2012-11-27 DigitalOptics Corporation Europe Limited Method and apparatus for red-eye detection using preview or other reference images
JP4746892B2 (ja) 2005-03-11 2011-08-10 Ｎｅｃディスプレイソリューションズ株式会社 画像表示装置のフレア補正回路、および画像表示装置のフレア補正方法
US9378685B2 (en) * 2009-03-13 2016-06-28 Dolby Laboratories Licensing Corporation Artifact mitigation method and apparatus for images generated using three dimensional color synthesis
JP6799385B2 (ja) * 2016-04-28 2020-12-16 キヤノン株式会社 撮像装置およびその制御方法
JP2018148318A (ja) * 2017-03-02 2018-09-20 キヤノン株式会社 画像処理装置およびその制御方法及びプログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001203933A (ja) * 2000-01-19 2001-07-27 Hitachi Ltd 撮像装置
JP2002077591A (ja) * 2000-09-05 2002-03-15 Minolta Co Ltd 画像処理装置および撮像装置
JP2005295513A (ja) * 2004-03-05 2005-10-20 Matsushita Electric Ind Co Ltd 画像信号処理方法、画像信号処理装置および画像表示装置
JP2012085083A (ja) * 2010-10-12 2012-04-26 Nikon Systems Inc 画像処理装置、撮像装置および画像処理プログラム
JP2012134630A (ja) * 2010-12-20 2012-07-12 Nikon Corp 画像処理方法、画像処理プログラム、画像処理装置、及び撮像装置
WO2017022513A1 (ja) * 2015-07-31 2017-02-09 ソニー株式会社 映像信号処理装置および映像信号処理方法、ならびに表示装置

Also Published As

Publication number Publication date
JP7369175B2 (ja) 2023-10-25
US20190378468A1 (en) 2019-12-12
JP2018148318A (ja) 2018-09-20
JP2022036179A (ja) 2022-03-04
US11328679B2 (en) 2022-05-10

Similar Documents

Publication Publication Date Title
JP7369175B2 (ja) 画像処理装置およびその制御方法及びプログラム
US9852499B2 (en) Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification
CN108668093B (zh) Hdr图像的生成方法及装置
JP5392560B2 (ja) 画像処理装置および画像処理方法
US9955131B2 (en) Projector and image display method
KR101629825B1 (ko) Ｈｄｒ 기능을 이용한 차량용 디스플레이 장치 및 방법
JP5901667B2 (ja) 画像処理装置及び方法、画像処理プログラム、撮像装置
CN106575434B (zh) 图像处理装置、摄像装置以及图像处理方法
JP2017138647A (ja) 画像処理装置、画像処理方法、映像撮影装置、映像記録再生装置、プログラム及び記録媒体
JP6412386B2 (ja) 画像処理装置およびその制御方法、プログラムならびに記録媒体
JP2019152910A (ja) 画像処理装置、画像処理方法及びプログラム
JP6611576B2 (ja) 画像処理装置および画像処理方法
JP5994301B2 (ja) 画像処理装置、情報処理装置、方法、プログラムおよび記録媒体
JP5410378B2 (ja) 映像信号補正装置および映像信号補正プログラム
CN105282419A (zh) 去噪声方法以及影像系统
JP2019047336A5 (enExample)
JP5365881B2 (ja) 画像処理装置、画像処理方法
JP5537995B2 (ja) 画像処理装置および画像処理プログラム
JP5103984B2 (ja) 画像処理装置、撮像装置、および画像処理プログラム
KR20160001582A (ko) 화상 처리 장치 및 화상 처리 방법
WO2018167898A1 (ja) 映像表示装置
JP7134766B2 (ja) 画像処理装置、画像処理方法及びプログラム
US12504813B2 (en) Eye-directed image signal processing
US20250117076A1 (en) Eye-directed image signal processing
JP2019140674A (ja) 画像処理装置、画像処理方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18761972

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18761972

Country of ref document: EP

Kind code of ref document: A1