WO2022094945A1

WO2022094945A1 - Method and apparatus for tone mapping, and computer usable medium storing software for implementing the method

Info

Publication number: WO2022094945A1
Application number: PCT/CN2020/127210
Authority: WO
Inventors: Yamamuro DAI
Original assignee: Guangdong Oppo Mobile Telecommunications Corp., Ltd.
Priority date: 2020-11-06
Filing date: 2020-11-06
Publication date: 2022-05-12
Also published as: CN116508326A

Abstract

A method for tone mapping on an image according to the present disclosure includes, acquiring an image, deciding whether a shooting condition is a backlight condition, when the shooting condition is decided not to be a backlight condition, calculating a luminance using camera parameters and selecting a tone curve according to the calculated luminance, when the shooting condition is decided to be a backlight condition, selecting a tone curve suitable for a backlight condition, and performing tone mapping on the image using the selected tone curve.

Description

METHOD AND APPARATUS FOR TONE MAPPING, AND COMPUTER USABLE MEDIUM STORING SOFTWARE FOR IMPLEMENTING THE METHOD

TECHNICAL FIELD

The present disclosure relates to the image processing technology field in general, and more specifically, to a method and an apparatus for tone mapping on an image, and a computer usable medium storing software for implementing the method.

BACKGROUND

In recent years, color imaging apparatuses have been developed and widely used in devices such as mobile phones, digital cameras, etc. Along with progresses in high-speed/large-capacity data communication, high-speed processors or high color quality is also required.

One of the technologies, a facial recognition system for face unlock of a smart phone, uses images; performing tone mapping on an original image is effective in increasing its success rate.

However, in certain conditions such as backlighting, a proper image may not be obtained.

According to conventional techniques, a histogram is used to calculate a luminance for selecting a tone curve used for tone mapping. However, this method is time consuming and not always effective.

As a result, it may be difficult to obtain a proper image in a limited time; thus it is not possible to increase the success rate of face unlock.

SUMMARY

The present disclosure aims to solve at least one of the technical problems explained above. Accordingly, the present disclosure needs to provide a method and an apparatus for tone mapping, and a computer usable medium storing software for causing a computer to implement the method for tone mapping.

According to one aspect of the disclosure, a method for tone mapping may comprise:

acquiring an image;

deciding whether a shooting condition is a backlight condition;

when the shooting condition is decided not to be a backlight condition, calculating a luminance using camera parameters, and selecting a tone curve according to the calculated luminance;

when the shooting condition is decided to be a backlight condition, selecting a tone curve suitable for a backlight condition; and

performing tone mapping on the image using the selected tone curve.

In some embodiments, the method may further comprise:

performing face unlock using the tone mapped image and deciding whether the face unlock is successful; and

displaying the decision on the face unlock.

In some embodiments, when deciding whether a shooting condition is a backlight condition, the method may further comprise:

setting a certain area on the image;

calculating a first average luminance of an inside of the certain area and a second average luminance of an outside of the certain area;

deciding whether a difference between the first and second average luminances is greater than a threshold value; and

when the difference is decided to be greater than the threshold value, deciding that the condition is a backlight condition, and when the difference is decided not to be greater than the threshold value, deciding that the condition is not a backlight condition.

In some embodiments, the certain area may be set in a manner that a center position of the certain area is located on a line whose direction is that of gravity.

In some embodiments, when the shooting condition may be decided not to be a backlight condition, the luminance is calculated using the following formula including camera parameters, namely F_value ², exposure time and ISO sensitivity:

where C is a calibration contrast.

In some embodiments, the image may be a still image or one of still images forming a moving image.

According to one aspect of the disclosure, an apparatus for tone mapping, the apparatus may comprise:

a camera unit configured to acquire an image;

a processor configured to input the current image, to decide whether a shooting condition is a backlight condition, when the shooting condition is decided not to be a backlight condition, to calculate a luminance using camera parameters and selecting a tone curve according to the calculated luminance, when the shooting condition is decided to be a backlight condition, to select a tone curve suitable for a backlight condition, and to perform tone mapping on the image using the selected tone curve; and

a display unit configured to display the tone mapped image.

In some embodiments, the processor may be further configured to perform face unlock using the tone mapped image, decide whether the face unlock is successful, and display the decision on the face unlock.

In some embodiments, when the processor decides whether the shooting condition is a backlight condition, the processor may be further configured to set a certain area on the image, to calculate a first average luminance of an inside of the certain area and a second average luminance of an outside of the certain area, to decide whether a difference between the first and second average luminances is greater than a threshold value, when the difference is decided to be greater than the threshold value, to decide that the condition is a backlight condition, and when the difference is decided not to be greater than the threshold value, to decide that the condition is not a backlight condition.

In some embodiments, the processor may set the area in a manner that a center position of the area is located on a line whose direction is that of gravity.

In some embodiments, when the shooting condition is decided not to be a backlight condition, the processor may calculate the luminance using the following formula including camera parameters, namely F_value ², exposure time and ISO sensitivity:

where C is a calibration contrast.

According to one aspect of the disclosure, a computer usable medium stores software for causing a computer to implement the above method for tone mapping.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects and advantages of embodiments of the present disclosure will become apparent and more readily appreciated from the following descriptions made with reference to the drawings, in which:

FIG. 1 is a flowchart illustrating a method for tone mapping according to a comparative example;

FIG. 2A is a photo of an example of an original image in a backlight condition;

FIG. 2B is a photo of an example of an image obtained by the method for tone mapping according to the comparative example;

FIG. 3 is a block diagram schematically showing a circuit configuration of an apparatus for tone mapping according to a first embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating a method for tone mapping according to a second embodiment of the present disclosure;

FIG. 5 is a flowchart illustrating a method for deciding whether a shooting condition is a backlight condition according to the second embodiment of the present disclosure;

FIG. 6 is a photo for illustrating a rectangular area for face unlock and a gravity direction;

FIG. 7A is a photo of an example of an original image in a backlight condition; and

FIG. 7B is a photo of an example of an image obtained by the method for tone mapping according to the first embodiment and the second embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detail and examples of the embodiments will be illustrated in the accompanying drawings. The same or similar elements and the elements having same or similar functions are denoted by like reference numerals throughout the descriptions. The embodiments described herein with reference to the drawings are explanatory and aim to illustrate the present disclosure, but shall not be construed to limit the present disclosure.

Before explaining a method for tone mapping on an image according to an example of the present disclosure, a method according to a comparative example, which does not detect a backlight condition, is explained for the sake of comparison.

The method for tone mapping according to the comparative example includes the following steps shown in FIG. 1.

In the step S11, a luminance is calculated by camera parameters, such as F_value, ISO sensitivity and/or exposure time.

In the step S12, a tone curve is selected according to the calculated luminance.

In the step S13, tone mapping is performed using the selected tone curve.

In an example of a photo taken in a backlight condition, a center area is dark and a surrounding area is much brighter than the center area, as illustrated in FIG. 2A.

In such a typical backlight condition, the center area may be too dark to recognize the face; thus a proper image cannot be obtained according to the above method.

Even if a backlight condition is detected in a conventional facial recognition system, a histogram is used to calculate the luminance, which is time consuming, as explained above.

An apparatus for tone mapping on an image according to a first example of the present disclosure is as follows.

Referring to FIG. 3, an apparatus 10 performing tone mapping for face unlock according to the first embodiment of the present disclosure includes a camera unit 11, a processor 12, a display unit 13 and a memory 14.

The camera unit 11 acquires an image under various shooting conditions, including a backlight condition. The image may be a still image or one of still images forming a moving image.

The processor 12 inputs the image from the camera unit 11, decides whether the shooting condition is a backlight condition, selects a proper tone curve in accordance with the decision, performs tone mapping on the acquired original image using the selected tone curve, and performs face unlock using the tone mapped image.

When the processor 12 decides that the shooting condition is a backlight condition, the processor 12 selects a tone curve suitable for a backlight scene. When the processor 12 decides that it is not a backlight condition, the processor 12 calculates the luminance using camera parameters, such as F_value, ISO sensitivity and/or exposure time, and selects a tone curve according to the calculated luminance.

When deciding whether the shooting condition is a backlight condition, the processor 12 calculates a first average luminance of an inside of a rectangular area where a face is supposed to exist, calculates a second average luminance of an outside of the rectangular area, decides that the shooting condition is a backlight condition when the difference between the first and second average luminances is greater than a threshold value, and decides that the shooting condition is not a backlight condition when the difference is not greater than a threshold value.

The processor 12 further performs tone mapping on the original image using the selected tone curve, performs face unlock using the tone mapped image, and decides whether face unlock is successful.

The display unit 13 receives the decision of face unlock from the processor 12 and displays it on its screen.

A method for tone mapping according to a second embodiment of the present disclosure includes the following steps shown in FIG. 4.

In the step S21, an image is acquired, under various conditions, including a backlight condition. The image may be a still image or one of still images forming a moving image

In the step S22, it is decided whether the shooting condition is a backlight condition.

If YES, namely, if it is a backlight condition, in the step S25, a proper tone curve suitable for a backlight scene is selected. If NO, namely, if it is not a backlight condition, in the step S23, a luminance is calculated using camera parameters, such as F_value, ISO sensitivity and/or exposure time. In the step S24, a tone curve is selected according to the calculated luminance.

In the step S26, tone mapping of the acquired (original) image is performed using the selected tone curve.

In the step S27, face unlock is performed using the tone mapped image, and it is decided whether the face unlock is successful.

In the step S28, the decision on the face unlock is displayed.

A method for deciding whether the shooting condition is a backlight condition is explained in detail referring to FIG. 5 as follows.

In the step S31, a rectangular area is set on the image acquired at the step S21 and displayed on the display unit 13. The rectangular area 21 is an area where a face is supposed to exist, as shown in FIG. 6. A center position of the rectangular area is not precisely a center of the whole image, and is supposed to be offset to an extent on a line whose direction is that of gravity. The gravity direction on the image can be determined by an acceleration sensor of a smartphone. The size of the rectangular area may be empirically set to about 1/4 width and 1/4 height of the whole image, for example.

In the step S32, a first average luminance of an inside of the rectangular area and a second average luminance of an outside of the rectangular area are calculated.

In the step S33, it is decided whether a difference between the first and second average luminances is greater than a threshold value.

As a first example, the threshold value is 50, the first average luminance is 10, and the second average luminance is 100.

The difference between the first and second average luminance is 90, thus the difference is greater than the threshold value. This condition is decided to be a backlight condition.

As a second example, the threshold value is 50, the first average luminance is 75, and the second average luminance is 100.

The difference between the first and second average luminances is 25, thus the difference is not greater than the threshold value. This condition is decided not to be a backlight condition.

As explained above, in the step S33, if a difference between the first and second average luminance is greater than the threshold value, the process moves on to the step S34 and the condition is decided to be a backlight condition. If not, the process moves on to the step S35 and the condition is decided not to be a backlight condition.

Further, if the condition is decided not to be a backlight condition, the luminance may be calculated using, for example, the following formula including camera parameters, namely F_value ², exposure time and ISO sensitivity:

where C is a calibration contrast.

A computer usable medium according to a third embodiment of the present disclosure is a medium storing a software for causing a computer or an apparatus having a configuration of the above first example of the present disclosure to implement the method for tone mapping, and the method of the above second example of the present disclosure.

According to the above apparatus for tone mapping according to the first embodiment, the method for tone mapping according to the second embodiment and the medium for tone mapping according to the third embodiment of the present disclosure, even if a shooting condition is a backlight condition, a proper image can be obtained by performing tone mapping using a tone curve suitable for the backlight condition, and success rate of face unlock can be increased.

FIG. 7A illustrates a photo of an example of an original image in a backlight condition. An inside of the center area where a face exists is much darker than an outside of the area.

FIG. 7B illustrates a photo obtained by the above first, second and third embodiments of the present disclosure. The inside of the center area where the face exists is brighter than that of FIG. 7A, thus it is possible to recognize the face. As a result, a proper image is obtained, and success rate of face unlock can be increased.

In the description of embodiments of the present disclosure, it is to be understood that terms such as "central" , "longitudinal" , "transverse" , "length" , "width" , "thickness" , "upper" , "lower" , "front" , "rear" , "back" , "left" , "right" , "vertical" , "horizontal" , "top" , "bottom" , "inner" , "outer" , "clockwise" and "counterclockwise" should be construed to refer to the orientation or the position as described or as shown in the drawings in discussion. These relative terms are only used to simplify the description of the present disclosure, and do not indicate or imply that the device or element referred to must have a particular orientation or must be constructed or operated in a particular orientation. Thus, these terms cannot be constructed to limit the present disclosure.

In addition, terms such as "first" and "second" are used herein for purposes of description and are not intended to indicate or imply relative importance or significance or to imply the number of indicated technical features. Thus, a feature defined as "first" and "second" may comprise one or more of these features. In the description of the present disclosure, "a plurality of" means “two or more than two” , unless otherwise specified.

In the description of embodiments of the present disclosure, unless specified or limited otherwise, the terms "mounted" , "connected" , "coupled" and the like are used broadly, and may be, for example, fixed connections, detachable connections, or integral connections; may also be mechanical or electrical connections; may also be direct connections or indirect connections via intervening structures; may also be inner communications of two elements which can be understood by those skilled in the art according to specific situations.

In the embodiments of the present disclosure, unless specified or limited otherwise, a structure in which a first feature is "on" or "below" a second feature may include an embodiment in which the first feature is in direct contact with the second feature, and may also include an embodiment in which the first feature and the second feature are not in direct contact with each other, but are in contact via an additional feature formed therebetween. Furthermore, a first feature "on" , "above" or "on top of" a second feature may include an embodiment in which the first feature is orthogonally or obliquely "on" , "above" or "on top of" the second feature, or just means that the first feature is at a height higher than that of the second feature; while a first feature "below" , "under" or "on bottom of" a second feature may include an embodiment in which the first feature is orthogonally or obliquely "below" , "under" or "on bottom of" the second feature, or just means that the first feature is at a height lower than that of the second feature.

Various embodiments and examples are provided in the above description to implement different structures of the present disclosure. In order to simplify the present disclosure, certain elements and settings are described in the above. However, these elements and settings are only by way of example and are not intended to limit the present disclosure. In addition, reference numbers and/or reference letters may be repeated in different examples in the present disclosure. This repetition is for the purpose of simplification and clarity and does not refer to relations between different embodiments and/or settings. Furthermore, examples of different processes and materials are provided in the present disclosure. However, it would be appreciated by those skilled in the art that other processes and/or materials may also be applied.

Reference throughout this specification to "an embodiment" , "some embodiments" , "an exemplary embodiment" , "an example" , "a specific example" or "some examples" means that a particular feature, structure, material, or characteristics described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. Thus, the appearances of the above phrases throughout this specification are not necessarily referring to the same embodiment or example of the present disclosure. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

Any process or method described in a flow chart or described herein in other ways may be understood to include one or more modules, segments or portions of codes of executable instructions for achieving specific logical functions or steps in the process, and the scope of a preferred embodiment of the present disclosure includes other implementations, in which it should be understood by those skilled in the art that functions may be implemented in a sequence other than the sequences shown or discussed, including in a substantially identical sequence or in an opposite sequence.

The logic and/or step described in other manners herein or shown in the flow chart, for example, a particular sequence table of executable instructions for realizing the logical function, may be specifically achieved in any computer readable medium to be used by the instructions execution system, device or equipment (such as a system based on computers, a system comprising processors or other systems capable of obtaining instructions from the instructions execution system, device and equipment executing the instructions) , or to be used in combination with the instructions execution system, device and equipment. As to the specification, "the computer readable medium" may be any device adaptive for including, storing, communicating, propagating or transferring programs to be used by or in combination with the instruction execution system, device or equipment. More specific examples of the computer readable medium comprise but are not limited to: an electronic connection (an electronic device) with one or more wires, a portable computer enclosure (a magnetic device) , a random access memory (RAM) , a read only memory (ROM) , an erasable programmable read-only memory (EPROM or a flash memory) , an optical fiber device and a portable compact disk read-only memory (CDROM) . In addition, the computer readable medium may even be a paper or other appropriate medium capable of printing programs thereon, this is because, for example, the paper or other appropriate medium may be optically scanned and then edited, decrypted or processed with other appropriate methods when necessary to obtain the programs in an electric manner, and then the programs may be stored in the computer memories.

It should be understood that each part of the present disclosure may be realized by the hardware, software, firmware or their combination. In the above embodiments, a plurality of steps or methods may be realized by the software or firmware stored in the memory and executed by the appropriate instructions execution system. For example, if it is realized by the hardware, likewise in another embodiment, the steps or methods may be realized by one or a combination of the following techniques known in the art: a discrete logic circuit having a logic gate circuit for realizing a logic function of a data signal, an application-specific integrated circuit having an appropriate combination logic gate circuit, a programmable gate array (PGA) , a field programmable gate array (FPGA) , etc.

Those skilled in the art shall understand that all or parts of the steps in the above exemplifying method of the present disclosure may be achieved by commanding the related hardware with programs. The programs may be stored in a computer readable storage medium, and the programs comprise one or a combination of the steps in the method embodiments of the present disclosure when run on a computer.

In addition, each function cell of the embodiments of the present disclosure may be integrated in a processing module, or these cells may be separate physical existence, or two or more cells are integrated in a processing module. The integrated module may be realized in a form of hardware or in a form of software function modules. When the integrated module is realized in a form of software function module and is sold or used as a standalone product, the integrated module may be stored in a computer readable storage medium.

The storage medium mentioned above may be read-only memories, magnetic disks, CD, etc.

Although embodiments of the present disclosure have been shown and described, it would be appreciated by those skilled in the art that the embodiments are explanatory and cannot be construed to limit the present disclosure, and changes, modifications, alternatives and variations can be made in the embodiments without departing from the scope of the present disclosure.

Claims

A method for tone mapping on an image, the method comprising:

acquiring an image;

deciding whether a shooting condition is a backlight condition;

when the shooting condition is decided not to be a backlight condition, calculating a luminance using camera parameters, and selecting a tone curve according to the calculated luminance;

when the shooting condition is decided to be a backlight condition, selecting a tone curve suitable for a backlight condition; and

performing tone mapping on the image using the selected tone curve.
The method according to claim 1, wherein the method further comprises:

performing face unlock using the tone mapped image and deciding whether the face unlock is successful; and

displaying the decision regarding the face unlock.
The method according to claim 1 or 2, wherein when deciding whether a shooting condition is a backlight condition, the method further comprises:

setting a certain area on the image;

calculating a first average luminance of an inside of the certain area and a second average luminance of an outside of the certain area;

deciding whether a difference between the first and second average luminances is greater than a threshold value; and

when the difference is decided to be greater than the threshold value, deciding that the condition is a backlight condition, and when the difference is decided not to be greater than the threshold value, deciding that the condition is not a backlight condition.
The method according to claim 3, wherein the certain area is set in a manner that a center position of the certain area is located on a line whose direction is that of gravity.
The method according to any one of claims 1 to 4, wherein when the shooting condition is decided not to be a backlight condition, the luminance is calculated using the following formula including camera parameters, namely F_value ², exposure time and ISO sensitivity:

where C is a calibration contrast.
The method according to any one of claims 1 to 5, wherein the image is a still image or one of still images forming a moving image.
An apparatus for tone mapping on an image, the apparatus comprising:

a camera unit configured to acquire an image;

a processor configured to input the current image, to decide whether a shooting condition is a backlight condition, when the shooting condition is decided not to be a backlight condition, to calculate a luminance using camera parameters and selecting a tone curve according to the calculated luminance, and when the shooting condition is decided to be a backlight condition, to select a tone curve suitable for a backlight condition, and to perform tone mapping on the image using the selected tone curve; and

a display unit configured to display the tone mapped image.
The apparatus according to claim 7, wherein the processor is further configured to perform face unlock using the tone mapped image, to decide whether the face unlock is successful, and to display the decision on the face unlock.
The apparatus according to claim 7 or 8, wherein when the processor decides whether the shooting condition is a backlight condition, the processor is further configured to set a certain area on the image, to calculate a first average luminance of an inside of the certain area and a second average luminance of an outside of the certain area, to decide whether a difference between the first and second average luminances is greater than a threshold value, when the difference is decided to be greater than the threshold value, to decide that the condition is a backlight condition, and when the difference is decided not to be greater than the threshold value, to decide that the condition is not a backlight condition.
The apparatus according to any one of claims 7 to 9, wherein the processor sets the area in a manner that a center position of the area is located on a line whose direction is that of gravity.
The apparatus according to any one of claims 7 to 10, wherein when the shooting condition is decided not to be a backlight condition, the processor calculates the luminance using the following formula including camera parameters, namely F_value ², exposure time and ISO sensitivity:

where C is a calibration contrast.
The apparatus according to any one of claims 7 to 11, wherein the image is a still image or one of still images forming a moving image.
A computer usable medium storing software for causing a computer to implement the method for tone mapping on an image according to any one of claims 1 to 6.