KR102030475B1 - Method and apparatus for transformation of display image according to ambient light - Google Patents

Method and apparatus for transformation of display image according to ambient light Download PDF

Info

Publication number
KR102030475B1
KR102030475B1 KR1020130006517A KR20130006517A KR102030475B1 KR 102030475 B1 KR102030475 B1 KR 102030475B1 KR 1020130006517 A KR1020130006517 A KR 1020130006517A KR 20130006517 A KR20130006517 A KR 20130006517A KR 102030475 B1 KR102030475 B1 KR 102030475B1
Authority
KR
South Korea
Prior art keywords
color
image
brightness
preset
displayed
Prior art date
Application number
KR1020130006517A
Other languages
Korean (ko)
Other versions
KR20140094161A (en
Inventor
안성주
김현지
조재영
고성제
정재윤
김소영
박정훈
유현국
추경호
Original Assignee
삼성전자주식회사
고려대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 삼성전자주식회사, 고려대학교 산학협력단 filed Critical 삼성전자주식회사
Priority to KR1020130006517A priority Critical patent/KR102030475B1/en
Publication of KR20140094161A publication Critical patent/KR20140094161A/en
Application granted granted Critical
Publication of KR102030475B1 publication Critical patent/KR102030475B1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/57—Control of contrast or brightness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/643—Hue control means, e.g. flesh tone control
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/06—Colour space transformation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/14—Detecting light within display terminals, e.g. using a single or a plurality of photosensors
    • G09G2360/144—Detecting light within display terminals, e.g. using a single or a plurality of photosensors the light being ambient light

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

A method of converting a display image according to an ambient light source, the method comprising: detecting illuminance of a surrounding environment; Changing brightness of an image to be displayed according to the detected illumination of the surrounding environment; Compensating the saturation of the image to be displayed using the detected illumination of the surrounding environment and a chromaticity coordinate of a predetermined white point, and the preset color range among the colors of the image compensated for saturation And a flare compensation process of converting out of color using colors within the preset color range.

Description

TECHNICAL AND APPARATUS FOR TRANSFORMATION OF DISPLAY IMAGE ACCORDING TO AMBIENT LIGHT

The present invention relates to an image processing technology, and to a method and apparatus for converting a display image according to an ambient light source in consideration of a usage environment of a mobile device.

The use of various types of mobile devices, such as smartphones, tablet PCs, and personal digital assistants (PDAs), has become commonplace. In display devices of mobile devices, various images are used for better color reproduction. Processing technology has been developed.

In image processing technology, recently, technologies that consider the use environment of a mobile device have been introduced, which is different from an electronic device used in a fixed place such as a TV or a desktop computer. This is because it is not limited to indoor environments.

In practice, the environment of use of the mobile device may be various environments having different intensity of light sources such as an external environment in which sunlight directly shines or an indoor environment with lighting. For example, when using a mobile device in a high light source, the user may perceive the same image differently depending on the ambient light source. Particularly, in the outdoor where the light source is high, since the luminance of the display device of the mobile device is relatively lower than that of the outdoor light source, the response value of the visual curve is lowered, and the image is darker than the indoor environment. This is because the human visual sensitivity curve (also called the response curve of cone cells) changes to adapt to the ambient light source.

Sumanta N. Pattanaik, Jack Tumblin, Hector Yee, and Donald P. Greenberg write “Time-dependent visual adaptation for fast realistic image display, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.47-54, July 2000” Referring to the contents disclosed in FIG. 1, FIG. 1 is a graph illustrating a change in visual sensitivity according to a change in intensity of an ambient light source in a light adaptation model. It can be seen that the response curve of the cone cells shifts to the right side. This means that the higher the ambient light source, the lower the response value (conical cell response) of the visual sensitivity curve.

As an example of image processing technology, considering the usage environment of a mobile device, MY Lee, CH Son, JM Kim, CH Lee, and YH Ha described “Illumination-Level adaptive color reproduction method with lightness adaptation and flare compensation for mobile display, Journal of Imaging Science and Technology, vol. 51 (1), pp. 4452, 2007 ”, as shown in Fig. 2, an algorithm for calculating the brightness of the ambient light measured by the illumination sensor and the flare reflected from the display and converting the image to be displayed based on the light adaptation model. Initiate.

Another example of an image processing technology in consideration of the use environment of a mobile device is Korean Patent Registration No. 10-0763239 (name; "Image Processing Apparatus and Method for Improving the Visibility of Displayed Image", Inventor; Kim In-ji, Ok Hyun-uk, Applicant; Samsung Electronics Co., Ltd., registration date; September 27, 2007), which is, as shown in Figure 3, to solve the problem of degradation of visibility of the display image generated when the ambient brightness is brighter than the image display device, And an image processing method. More specifically, using a preference evaluated from a plurality of users, to obtain a luminance-contrast model curve representing the optimal contrast value for a specific luminance for each brightness of the external light to adjust the brightness of the image and to adjust the saturation value It is about.

Currently, various image processing technologies have been proposed in consideration of the use environment of the mobile device as described above. Image processing techniques applying these techniques, for example, in the case of the Korean Patent Registration No. 10-0763239, by defining a brightness-contrast model from the experimental value, inaccurate results may occur, a number of experiments per mobile device It is inconvenient to determine the evaluated preferences from the user. In addition, conventionally proposed image processing techniques are limited in increasing saturation as the luminance value increases, depending on the characteristics of the expressible color range.

Accordingly, an object of the present invention is to provide a method and apparatus for converting a display image according to an ambient light source to solve a problem that an image is dark due to a decrease in visibility when using a mobile device in an environment with high illumination. In providing.

Another object of the present invention is to convert the display image according to the ambient light source to solve the problem of the conventional image processing technology, which is limited to increase the saturation as the luminance value increases according to the characteristics of the color range that can be expressed and In providing a device.

According to one aspect of the present invention, a method of converting a display image according to an ambient light source, the method comprising: detecting an illuminance of a surrounding environment; Changing brightness of an image to be displayed according to the detected illumination of the surrounding environment; Compensating the saturation of the image to be displayed using the detected illumination of the surrounding environment and a chromaticity coordinate of a predetermined white point, and the preset color range among the colors of the image compensated for saturation And a flare compensation process of converting out of color using colors within the preset color range.

According to another aspect of the present invention, an apparatus for converting a display image according to an ambient light source, the apparatus comprising: a display unit for displaying an execution image of an application program, an operation state, and a menu state; An illuminance sensor detecting an illuminance of the surrounding environment; Controlling the display unit and the illuminance sensor, changing brightness of an image to be displayed according to the detected illuminance of the surrounding environment, illuminance of the detected surrounding environment and a predetermined white point ( Compensation using a chromaticity coordinate of a white point, and controls a flare compensation operation of converting a color out of a preset color range among the colors of the saturation compensated image using a color within the preset color range. It characterized in that it comprises a control unit.

As described above, by using the method and apparatus for converting the display image according to the ambient light source according to the present invention, by providing a color representation suitable for the use environment of the mobile device (readability) of the display in the outdoor environment To increase. In other words, by using the method and apparatus for converting the display image according to the ambient light source according to the present invention, it is possible to improve the brightness of the image in consideration of the ambient light source and the flare phenomenon, and to reduce the brightness contrast ratio during brightness enhancement Make it work. In addition, it provides a user parameter that can adjust the specific gravity between brightness enhancement and brightness contrast ratio, so that it can be applied to various environments and users, and the saturation reduced by the flare phenomenon is compensated by the reduced size, so that the color of the original image can be adjusted even in a specific environment. To reproduce.

1 is a graph illustrating a change in visual sensitivity according to a change in intensity of an ambient light source in a light adaptation model according to an exemplary embodiment of the prior art;
2 is a conversion flowchart of an image to be displayed, according to an ambient light source, according to another embodiment of the prior art;
3 is a block diagram of an image processing apparatus according to another embodiment of the prior art;
4 is a block diagram of a mobile device to which a device for converting a display image according to an ambient light source is applied according to an embodiment of the present invention;
5 is a flowchart illustrating a conversion operation of a display image according to an ambient light source according to an exemplary embodiment of the present invention.
6 is a flowchart of an operation of changing brightness of an image to be displayed according to an exemplary embodiment of FIG. 5.
FIG. 7 is a detailed step-by-step result graph of an operation of changing brightness of an image to be displayed according to an embodiment of FIG. 6;
8 is a flowchart of a flare compensation operation according to an embodiment of FIG. 5.
9 is an exemplary converted image illustrating a result of a conversion operation of a display image according to an ambient light source according to an exemplary embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, specific matters such as a brightness conversion function, a user parameter, etc. are shown, which are provided to help a more general understanding of the present invention, and such specific matters may be modified or changed within the scope of the present invention. Will be apparent to those of ordinary skill in the art.

4 is a block diagram of a smartphone among mobile devices to which a device for converting a display image according to an ambient light source is applied according to an embodiment of the present invention.

Referring to FIG. 4, the mobile device 100 includes a display 190 and a display controller 195. In addition, the mobile device 100 may include a control unit 110, a mobile communication module 120, a sub communication module 130, a multimedia module 140, a camera module 150, a GPS module 155, and an input / output module ( 160, a sensor module 170, a storage unit 175, and a power supply unit 180. The sub communication module 130 includes at least one of the wireless LAN module 131 and the short range communication module 132, and the multimedia module 140 includes the broadcast communication module 141, the audio play module 142, and the video play module. 143 at least one. The camera module 150 includes at least one of the first camera 151 and the second camera 152, and the input / output module 160 includes a button 161, a microphone 162, a speaker 163, and vibrations. At least one of a motor 164, a connector 165, and a keypad 166. Hereinafter, the case where the display unit 190 and the display controller 195 are a touch screen and a touch screen controller will be described as an example.

The mobile communication module 120 allows the mobile device 100 to be connected to an external device through mobile communication using at least one or a plurality of antennas (not shown) under the control of the controller 110. The mobile communication module 120 may make a voice call or video call with a mobile phone (not shown), a smartphone (not shown), a tablet PC or another device (not shown) having a phone number input to the mobile device 100. Send / receive wireless signals for text messages (SMS) or multimedia messages (MMS).

The sub communication module 130 may include at least one of the WLAN module 131 and the short range communication module 132.

The WLAN module 131 may be connected to the Internet at a place where a wireless access point (AP) (not shown) is installed under the control of the controller 110. The WLAN module 131 supports a WLAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The short range communication module 132 may wirelessly perform short range communication between the mobile devices 100 under the control of the controller 110.

The mobile device 100 may include at least one of the mobile communication module 120, the wireless LAN module 131, and the short range communication module 132 according to performance. For example, the mobile device 100 may include a combination of the mobile communication module 120, the wireless LAN module 131, and the short range communication module 132 according to performance.

The multimedia module 140 may include a broadcast communication module 141, an audio play module 142, or a video play module 143. The broadcast communication module 141 may control a broadcast signal (eg, a TV broadcast signal, a radio broadcast signal or a data broadcast signal) transmitted from a broadcast station through a broadcast communication antenna (not shown) under the control of the controller 110 and the broadcast unit information. For example, an electric program guide (EPS) or an electric service guide (ESG) may be received. The audio play module 142 may play a digital audio file (eg, a file extension of mp3, wma, ogg, or wav) stored or received under the control of the controller 110. The video playback module 143 may play a digital video file (eg, a file extension of mpeg, mpg, mp4, avi, mov, or mkv) stored or received under the control of the controller 110. The video play module 143 may play a digital audio file.

The multimedia module 140 may include an audio play module 142 and a video play module 143 except for the broadcast communication module 141. In addition, the audio playback module 142 or the video playback module 143 of the multimedia module 140 may be included in the controller 110.

The camera module 150 may include at least one of the first camera 151 and the second camera 152 for capturing a still image or a video under the control of the controller 110.

The GPS module 155 receives radio waves from a plurality of GPS satellites (not shown) on Earth's orbit and uses a time of arrival from the GPS satellites (not shown) to the mobile device 100. To calculate the location of the mobile device 100.

The input / output module 160 may include at least one of a plurality of buttons 161, a microphone 162, a speaker 163, a vibration motor 164, a connector 165, and a keypad 166.

Button 161 may be formed on the front, side or rear of the housing of the mobile device 100, the power / lock button (not shown), volume buttons (not shown), menu button, home button, back It may include at least one of a back button and a search button 161.

The microphone 162 generates an electric signal by receiving a voice or sound under the control of the controller 110.

The speaker 163 may control various signals (eg, wireless signals, broadcast signals, etc.) of the mobile communication module 120, the sub communication module 130, the multimedia module 140, or the camera module 150 under the control of the controller 110. Sound corresponding to a digital audio file, a digital video file, or a photo capture) may be output to the outside of the mobile device 100. The speaker 163 may output a sound corresponding to a function performed by the mobile device 100 (eg, a button operation sound corresponding to a phone call, or a call connection sound). One or more speakers 163 may be formed at appropriate locations or locations of the housing of the mobile device 100.

The vibration motor 164 may convert an electrical signal into mechanical vibration under the control of the controller 110. For example, when the mobile device 100 in the vibration mode receives a voice call from another device (not shown), the vibration motor 164 operates. One or more may be formed in the housing of the mobile device 100. The vibration motor 164 may operate in response to a touch operation of a user who touches the touch screen 190 and a continuous movement of the touch on the touch screen 190.

The connector 165 may be used as an interface for connecting the mobile device 100 to an external device (not shown) or a power source (not shown). The mobile device 100 transmits data stored in the storage unit 175 of the mobile device 100 to an external device (not shown) through a wired cable connected to the connector 165 under the control of the controller 110. Alternatively, data may be received from an external device (not shown). In addition, the mobile device 100 may receive power from a power source (not shown) through a wired cable connected to the connector 165, or charge a battery (not shown) using the power source.

The keypad 166 may receive a key input from a user for control of the mobile device 100. The keypad 166 includes a physical keypad (not shown) formed on the mobile device 100 or a virtual keypad (not shown) displayed on the touch screen 190. The physical keypad (not shown) formed in the mobile device 100 may be excluded according to the performance or structure of the mobile device 100.

An earphone (not shown) may be inserted into an earphone connecting jack 167 to be connected to the mobile device 100.

The power supply unit 180 may supply power to one or a plurality of batteries (not shown) disposed in the housing of the mobile device 100 under the control of the controller 110. One or more batteries (not shown) power the mobile device 100. In addition, the power supply unit 180 may supply power input from an external power source (not shown) to the mobile device 100 through a wired cable connected to the connector 165. In addition, the power supply unit 180 may supply power to the mobile device 100 that is wirelessly input from an external power source through a wireless charging technology.

The touch screen 190 may receive a user's manipulation and display an execution image, an operation state, and a menu state of an application program. That is, the touch screen 190 may provide a user interface corresponding to various services (eg, a call, data transmission, broadcasting, photographing). The touch screen 190 may transmit an analog signal corresponding to at least one touch input to the user interface to the touch screen controller 195. The touch screen 190 may receive at least one touch through a user's body (eg, a finger including a thumb) or a touchable input means (eg, a stylus pen) such as an electronic pen. In addition, the touch screen 190 may receive a continuous movement of one touch among at least one touch. The touch screen 190 may transmit an analog signal corresponding to continuous movement of an input touch to the touch screen controller 195.

In addition, in the present invention, the touch is not limited to the direct contact between the touch screen 190 and the user's body or touchable input means, and may include non-contact. The detectable interval on the touch screen 190 may be changed according to the performance or structure of the mobile device 100. In particular, the touch screen 190 may include a touch event by contact with a user's body or touchable input means, A value detected by the touch event and the hovering event (eg, a current value, etc.) may be output differently so that the input to the non-contact state (eg, a hovering) event can be detected separately. In addition, the touch screen 190 may differently output a detected value (eg, a current value) according to the distance between the space where the hovering event occurs and the touch screen 190.

The touch screen 190 may be implemented by, for example, a resistive method, a capacitive method, an infrared method, or an ultrasonic wave method.

Meanwhile, the touch screen controller 195 converts an analog signal received from the touch screen 190 into a digital signal (for example, X and Y coordinates) and transmits the same to the controller 110. The controller 110 may control the touch screen 190 by using the digital signal received from the touch screen controller 195. For example, the controller 110 may select a shortcut icon (not shown) displayed on the touch screen 190 or execute a shortcut icon (not shown) in response to a touch event or a hovering event. In addition, the touch screen controller 195 may be included in the controller 110.

In addition, the touch screen controller 195 may detect a value (eg, a current value) output through the touch screen 190 to check the distance between the space where the hovering event occurs and the touch screen 190, and confirm The converted distance value may be converted into a digital signal (eg, Z coordinate) and provided to the controller 110.

In addition, the touch screen 190 may include at least two touch screen panels each capable of sensing a touch or proximity of the user's body and the touchable input means so as to simultaneously receive input by the user's body and the touchable input means. It may include. The at least two touch screen panels provide different output values to the touch screen controller 195, and the touch screen controller 195 recognizes values input from the at least two touch screen panels to be different from each other. Whether the input is input by a user's body or by a touchable input means can be distinguished.

The sensor module 170 includes at least one sensor that detects a state of the mobile device 100. For example, the sensor module 170 may be a proximity sensor that detects whether the user approaches the mobile device 100, or an operation of the mobile device 100 (eg, rotation of the mobile device 100, mobile device 100). Motion sensor (not shown) that detects acceleration or vibration), geo-magnetic sensor (not shown) that detects a point of the compass using the Earth's magnetic field, and the direction of gravity It may include a gravity sensor (Gravity Sensor) for detecting the, Altimeter for detecting the altitude by measuring the pressure of the atmosphere. The sensor of the sensor module 170 may be added or deleted according to the performance of the mobile device 100. At least one sensor may detect a state, generate a signal corresponding to the detection, and transmit the signal to the controller 110.

In addition, the sensor module 170 may include an illumination sensor 171 for detecting the amount of light around the mobile device 100 according to the characteristics of the conversion device of the display image according to the ambient light source of the present invention. The illumination sensor 171 may detect the illumination of the surrounding environment, generate a signal corresponding to the detection, and transmit the generated signal to the controller 110.

The storage unit 175 is a mobile communication module 120, a sub communication module 130, a multimedia module 140, a camera module 150, a GPS module 155, an input / output module under the control of the controller 110. In operation 160, the input / output signal or data corresponding to the operation of the sensor module 170 and the touch screen 190 may be stored. The storage unit 175 may store a control program and applications for controlling the mobile device 100 or the controller 110.

The term " storage " refers to a memory card (not shown) mounted on the storage unit 175, the ROM 112 in the controller 110, the RAM 113, or the mobile device 100 (e.g., an SD card, a memory). Sticks). The storage unit may include a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

The controller 110 may store a signal or data input from an external device (ROM) 112 and a device 100 in which the CPU 111, a control program for controlling the apparatus 100, and the apparatus 100 may be stored. It may include a RAM 113 used as a storage area for the operation to be performed. The CPU 111 may include a single core, dual cores, triple cores, or quad cores. The CPU 111, the ROM 112, and the RAM 113 may be connected to each other through an internal bus.

The controller 110 may include a mobile communication module 120, a sub communication module 130, a multimedia module 140, a camera module 150, a GPS module 155, an input / output module 160, and a sensor module 170. , The storage unit 175, the power supply unit 180, the touch screen 190, and the touch screen controller 195 may be controlled.

In addition, the control unit 111 changes the brightness of the image to be displayed according to the illuminance of the surrounding environment detected by the illuminance sensor 171 according to the feature of the conversion of the display image according to the ambient light source of the present invention, and The chroma of the image to be displayed is compensated by using the illumination of the surrounding environment detected by the illumination sensor 171 and the chromaticity coordinate of a predetermined white point. A flare compensation operation of converting a color out of a preset color range by using a color within the preset color range may be controlled.

FIG. 5 is a flowchart illustrating an operation of converting a display image according to an ambient light source according to an exemplary embodiment of the present invention. When the mobile device 100 is used outdoors with a high light source, the image may appear dark due to the sun and may be difficult to recognize the image. This section describes the operation to solve the problem. When the operation of FIG. 5 is largely divided into two types, the first operation is to improve brightness of an image by reflecting illuminance of the surrounding environment. Second, the saturation of the image is reduced due to the flare phenomenon in which the light emitted from the ambient light is reflected on the display, and the saturation of the reduced image is compensated using the illumination of the surrounding environment and the chromaticity coordinates of the preset white point. In addition, a color out of the range of colors that can be expressed among the colors of the image that has been compensated for saturation is generated by the saturation compensation of the image. The operation of converting such colors using colors within the range of colors that can be expressed. In this case, the color range that can be expressed is preset for each display unit 190 of each mobile device 100.

Referring to FIG. 5, first, in step 202, the illuminance of the surrounding environment of the mobile device 100 is detected using the illuminance sensor 171 of the mobile device 100. Thereafter, in step 204, the RGB value of the image to be displayed on the display unit 190 is converted into an XYZ color display system. In step 206, the XYZ color display system is converted into a CIE LAB color space. Thereafter, in step 208, only the brightness component of the image to be displayed on the display unit 190 in the CIE LAB color space is extracted, and reflects the illuminance of the surrounding environment detected by the illuminance sensor 171, the display unit 190 is displayed on the display unit 190 As a step of changing the brightness of the image to be described, a detailed description will be described later. Thereafter, in step 210, the flare compensation operation is performed. That is, the saturation of the image to be displayed on the display unit 190 is compensated by using the illuminance of the surrounding environment detected by the illuminance sensor 171 and the chromaticity coordinates of the predetermined white point, and by using the color within the preset color range. An operation of converting a color out of a preset color range among the colors of the image compensated for saturation is performed. Operation of step 210 is described in more detail below. After step 210, step 212 converts the CIE LAB color space into an XYZ color indicator, and then, in step 214, converts the XYZ color indicator into an RGB value. After step 214, in step 216, an image converted to RGB values is output.

Referring to Figure 6, the operation of changing the brightness of the image of step 208 described above in detail.

First, in step 2081, according to the illumination of the ambient environment detected by the illumination sensor 171, the luminance adaptation model (Luminance Adaptation Model) to determine the degree of response of the cone cells (Cone response) forming the optic nerve of the eye The cone response curve is generated and linearized to generate the cone response curve. In this case, in order to improve the brightness of the image, a function for performing linearization of the cone response curve is called a lightness transform function (X L ).

The relationship between the illumination of the ambient light source, the cone response curve of the luminance adaptation model, and the color recognition of the image will be described in detail. For example, as the illumination of the surrounding environment increases, the display 190 of the mobile device 100 is expressed. Within the range of possible luminance, the response curves of the cone cells become gentle. As a result, in the color recognition of the image displayed on the display unit 190 in a high illumination environment, the human eye recognizes the color as less colorful. Accordingly, it is necessary to linearize the response curve of the cone cells within the expressible luminance range of the display unit 190 so that the human eye can recognize the color of the image more colorfully in the high illumination environment. However, the luminance range that can be expressed by the display unit 171 is set in advance for each display unit 171 of the mobile device 100.

In operation 2081, when the brightness of the image is improved, a problem that the brightness contrast ratio of the image falls in a high illumination environment occurs. In order to solve this problem, it is necessary to improve the brightness contrast ratio of the image, where the brightness contrast ratio of the image refers to the contrast ratio of the brightness component when the image is expressed in the CIE LAB color space. Accordingly, in step 2085 after step 2081, in order to improve the brightness contrast ratio of the image reduced by the brightness enhancement of the image, histogram equalization of the contrast ratios of the brightness components of the CIE LAB color space is performed. Here, the function of improving the brightness contrast ratio is called a contrast enhancement function (X H ).

In order to reflect the brightness of the user's preferred image by adjusting the specific gravity of the brightness enhancement and the brightness contrast ratio enhancement of the image, after step 2085, the brightness conversion function and the contrast ratio enhancement function are combined in step 2089. Optimized image brightness is introduced to optimize the brightness of the image. To this end, a user parameter that can be specified by the user enters an input, and this parameter adjusts the degree of brightness enhancement of the image and the degree of contrast enhancement of the image. That is, in step 2089, when the preset user parameter is input, the brightness of the image to be displayed on the display unit 190 is optimized by adjusting the specific gravity of the linearization of the cone response curve and the histogram smoothing of the contrast ratio of the brightness components. In this case, a function of inputting a user parameter and optimizing by reflecting the user's preferred specific gravity is called a brightness enhancement function (L '), and a calculation formula of the brightness enhancement function is shown in Equation 1 below.

Figure 112013005723588-pat00001

At this time,

Figure 112013005723588-pat00002
Is a user parameter that can be selectively input by the user, and a default value may be preset. X is also the brightness component of the image to be displayed in the CIE LAB color space. X L is a function of linearizing the cone cell response curve of the luminance compliance model, and x H is a function of histogram smoothing of the contrast ratio of the brightness components. In addition, W is a 101 x 101 diagonal matrix, where each (i, j) element of the matrix is the ratio of the number of colors that can be represented at the brightness level of i in the CIE LAB color space and the number of colors that can be represented at the brightness level of j. T represents a transpose matrix. L 'is a value for optimizing the brightness of the image to be displayed.

Referring to FIG. 7, when the illuminance value of the surrounding environment is 1600 cd / m 2 , and the luminance range that can be expressed by the display unit 190 is about −2 to 2.6 Log 10 (cd / m 2 ), FIG. Detailed results of the operation of changing the brightness of an image are as follows. When the illuminance value of the ambient light source is 1600 cd / m 2 , the response curve of the cone cells of the luminance compliance model is shown in FIG. 7A. FIG. 7A illustrates a luminance range that can be expressed by the display unit 190, and is within -2 to 2.6 Log 10 (cd / m 2 ) that is a luminance range that can be expressed by the display unit 171. Linearizing the response curves of the cone cells, the graph is as shown in Figure 7 (a3). In this case, the graph of the brightness conversion function for linearizing the response curve of the cone cells is shown in FIG. In addition, the graph of the contrast ratio enhancement function for improving the brightness contrast ratio of the reduced image is shown in FIG. 7C. FIG. 7D is a graph of a brightness enhancement function according to a user parameter value for adjusting specific gravity of the brightness conversion function and the contrast ratio enhancement function. According to the above-described calculation, the graph of the brightness enhancement function is a user parameter value. When 0.1, the shape is similar to the curve d1 of the brightness conversion function, and when the user parameter value is 1, the shape is similar to the curve d2 of the contrast enhancement function.

Referring to FIG. 8, the flare compensation operation of step 210 will be described in detail as follows.

First, in step 2101, the CIE LAB color space is converted into an XYZ color display system.

In operation 2103, the chroma value is compensated by applying the chromaticity coordinates of the predetermined white point and the preset reflectance of the display unit 171 in the XYZ color display system.

In this case, the chromaticity coordinates of the preset white point may be set to the chromaticity coordinate values of the white point of the D65 daylight environment as a default value, and the coordinate values of x amb and y amb may be set to 0.31292 and 0.32933, respectively. D65 is a standard light source defined by the International Commision on Illumination (CIE). D65 is a standard light source representing an ambient light source in daylight, and the present invention considers an environment with high illuminance of the ambient light source (day environment), and thus, in the present invention, the chromaticity coordinate of the white point of D65 may be set as a basic coordinate value. .

In addition, the chromaticity coordinates of the preset white point may be set by the user. For example, the chromaticity coordinates of the white point according to various environments using the mobile device 100 are stored in advance, and various environments are prepared in advance, so that the user may select from a separate setting menu. Accordingly, when the user selects one of various environments, the chromaticity coordinate value of the white point corresponding to the environment selected by the user may be set. Various environments at this time may be, for example, D65, D55, and F5 with reference to the white point recommended by the CIE. Referring to the white point recommended by CIE, the white point of D55 in the late afternoon light environment (x amb = 0.33242, y amb = 0.34743) and F5 (x amb = 0.31379, y in the indoor environment where the ambient light is high) amb = 0.34531). In addition, the various environments that are prepared in advance may be set in the form of an icon, for example, to receive the usage environment of the mobile device 100 from the user on the display screen.

Compensation of the saturation value in step 2103 calculates the flare level according to the illumination of the surrounding environment by determining the cause of the saturation decrease as a flare phenomenon, and calculates the amount of saturation reduced from the calculated flare degree to the saturation of the image. Apply the addition method. If this is expressed as an equation, it is expressed as Equation 2 below.

Figure 112013005723588-pat00003

At this time,

Figure 112013005723588-pat00004
Are X, Y, and Z values in the XYZ color display system, R is a preset reflectance of the display unit 190, and x amb and y amb are coordinate values of the white point. Also,
Figure 112013005723588-pat00005
Is a value that compensates for the saturation value.

After 2103, in step 2105, the XYZ color indicator is converted back into the CIE LAB color space.

Thereafter, in step 2107, it is determined whether there is a color outside the color range in the converted CIE LAB color space. This is because a color outside the range of the expressible color may occur in the process of compensating the saturation of step 2103. If a color out of the range of the expressible color occurs, step 2109 is performed and the range of the expressible color is determined. If no color deviation occurs, the operation of converting the CIE LAB color space of step 212 of FIG. 5 to an XYZ color display system is performed.

In the CIE LAB color space, if there is a color of an image that is outside the preset color range, in step 2109, in order to convert the color of the image that is outside the preset color range into a color that can be expressed, a preset color among the colors within the preset color range is set. The color having the same color (HUE) value at the position closest to the color of the image out of the color range is detected, and the color of the image out of the preset color range is mapped to the detected color. If this is expressed as an equation, it is expressed as Equation 3 below.

Figure 112013005723588-pat00006

In this case, x e is a color outside the preset color range in the CIE LAB color space, and x is a basic color of the CIE LAB color space. Also,

Figure 112013005723588-pat00007
Is a color that can be expressed, and a color having an image color outside a preset color range and having the same color (HUE) value at a position closest to the color of the image outside the preset color range among the detected color ranges. Is the color mapped with.

Referring to FIG. 9, a converted image showing a result of a display image conversion operation according to an ambient light source according to an embodiment of the present invention may be checked. 9 (a) is an original image before the present invention is applied, and FIGS. 9 (b), 9 (c), and 9 (d) respectively correspond to input values of different user parameters. The converted image showing the operation result of.

As described above, the configuration and operation according to an embodiment of the present invention can be made. Meanwhile, in the above description of the present invention, specific embodiments have been described, but in addition to the various embodiments of the present invention, various modifications and changes may be made. Can be.

For example, the ambient light source, as well as the sun, may be a light source by a variety of illumination, in the environment having a variety of light sources in the outdoor or indoor, it is possible to apply the method and apparatus for converting the display image according to the ambient light source of the present invention Do.

In addition, the brightness change operation and the flare compensation operation of the image of the present invention may operate sequentially as described above, and may also perform the brightness change operation of the image after first performing the flare compensation operation. In addition, the brightness change operation and the flare compensation operation of the image of the present invention may be simultaneously operated independently.

In addition, in the above description of FIG. 5, an operation of converting an RGB value of step 204 to an XYZ color display system and converting an XYZ color display system of step 206 to a CIE LAB color space is disclosed, but the color space is a CIE LAB color space. In the state, when applying the method and apparatus for converting the display image according to the ambient light source of the present invention, steps 204 and 206 may be omitted.

It will also be appreciated that embodiments of the present invention may be implemented in the form of hardware, software or a combination of hardware and software. Such any software may be, for example, non-volatile storage such as storage, such as ROM, whether erasable or rewritable, or memory such as, for example, RAM, memory chips, devices or integrated circuits, or For example, it can be stored in a storage medium that can be optically or magnetically recorded such as CD, DVD, magnetic disk or magnetic tape, and can be read by a machine (eg, a computer). It will be appreciated that the memory that can be included in the portable terminal is an example of a machine-readable storage medium suitable for storing a program or programs containing instructions for implementing embodiments of the present invention. Accordingly, the present invention includes a program comprising code for implementing the apparatus or method described in any claim herein and a machine-readable storage medium storing such a program. In addition, such a program may be transferred electronically through any medium, such as a communication signal transmitted via a wired or wireless connection, and the present invention includes equivalents thereof as appropriate.

Claims (13)

In the conversion method of the display image according to the ambient light source,
Detecting illuminance of the surrounding environment;
Changing a brightness of an image to be displayed by using a cone response curve of a luminance adaptation model based on the detected illumination of the surrounding environment;
Compensating the saturation of the image to be displayed using the detected illumination of the surrounding environment and a chromaticity coordinate of a predetermined white point, and the preset color range among the colors of the image compensated for saturation A flare compensation process for converting out of color using colors within the preset color range,
The process of changing the brightness of the image to be displayed,
Generating the cone cell response curve of the luminance compliance model according to the detected ambient light intensity and linearizing it within a preset expressible brightness range;
Performing histogram equalization on the contrast ratio of the brightness components of the image to be displayed in the CIE LAB color space;
And optimizing the brightness of the image to be displayed by adjusting the specific gravity of the linearization of the cone cell response curve and the histogram smoothing of the contrast ratio of the brightness components according to input of a preset user parameter. Display image conversion method according to. delete The method of claim 1, wherein the CIE LAB color space,
And converting the RGB value of the image to be displayed into an XYZ color display system and converting the converted XYZ color display system into a CIE LAB color space. The method of claim 1, wherein the optimizing the brightness of the image to be displayed comprises:
Method of converting the display image according to the ambient light source, characterized in that it is performed by the following equation (1).
[Equation 1]
Figure 112019065392278-pat00008

In Equation 1, L 'is a value for optimizing the brightness of the image to be displayed,
x is the brightness component of the image to be displayed in the CIE LAB color space,
x L is a function to linearize the cone cell response curve of the luminance compliance model,
x H is a function to histogram smooth the contrast ratio of the brightness components,
Figure 112019065392278-pat00009
Is the user parameter,
W is a 101 x 101 diagonal matrix, where each (i, j) element of the matrix is the number of colors that can be represented at the brightness level of i in the CIE LAB color space and the number of colors that can be represented at the brightness level of j Of rain,
T represents the transpose matrix. The method of claim 1, wherein the chromaticity coordinates of the preset white point,
Method of converting the display image according to the ambient light source, characterized in that can be set by the user. The method of claim 1, wherein the flare compensation process,
Converting the CIE LAB color space into an XYZ color indicator;
Compensating for the chroma value of the image to be displayed by applying the chromaticity coordinates of the white point and a preset display reflectance in the converted XYZ color display system;
Converting the XYZ color indicator to a CIE LAB color space;
In the converted CIE LAB color space, when there is a color of an image out of the preset color range, the same color (HUE) at the position closest to the color of the image out of the preset color range among colors within the preset color range (HUE) And detecting a color having a value of) and mapping a color of an image that is out of the preset color range with the detected color. The method of claim 6, wherein the compensating of the saturation value comprises:
The conversion method of the display image according to the ambient light source, characterized in that it is performed by the following equation (2).
[Equation 2]
Figure 112013005723588-pat00010

In Equation 2,
Figure 112013005723588-pat00011
Is a value that compensates for the saturation value,
Figure 112013005723588-pat00012
Is the X, Y, Z value in the XYZ color display system,
R is the preset display reflectance,
x amb and y amb are chromaticity coordinate values of the white point. The method of claim 6, wherein the mapping step,
The conversion method of the display image according to the ambient light source, characterized in that it is performed by the following equation (3).
[Equation 3]
Figure 112019503324348-pat00013

In Equation 3,
Figure 112019503324348-pat00014
Is a color obtained by mapping a color of an image which is outside the preset color range with the detected color,
x is the base color of the CIE LAB color space,
x e is a color outside the preset color range in the CIE LAB color space,
However, the hue of x and x e is the same. The method of claim 6, wherein after the mapping step,
Converting the CIE LAB color space into an XYZ color indicator;
And converting the converted XYZ color display system into an RGB value. In the converting device of the display image according to the ambient light source,
A display unit displaying an execution image of the application program, an operation state, and a menu state;
An illuminance sensor detecting an illuminance of the surrounding environment;
Changing the brightness of an image to be displayed by controlling the display unit and the illuminance sensor and using a cone response curve of a luminance adaptation model based on the detected illuminance of the surrounding environment; And a saturation of the image to be displayed is compensated by using the detected illumination of the surrounding environment and a chromaticity coordinate of a predetermined white point, and a preset color range among the colors of the image that compensates for the saturation. And a controller configured to control a flare compensation operation of converting a color deviating from the color by using a color within the preset color range.
The control unit,
The cone cell response curve of the luminance adaptation model is generated and linearized within a preset expressible brightness range according to the detected ambient light intensity, and in the CIE LAB color space, a histogram of the contrast ratio of the brightness components of the image to be displayed. Histogram equalization, and adjusting the specific gravity of the histogram smoothing of the linearization of the cone response curve and the contrast ratio of the brightness components according to input of a preset user parameter, thereby controlling to optimize the brightness of the image to be displayed. Device for converting the display image according to the ambient light source further comprises. delete The method of claim 10, wherein the CIE LAB color space,
And converting the RGB value of the image to be displayed into an XYZ color display system and converting the converted XYZ color display system into a CIE LAB color space. The method of claim 10, wherein the flare compensation operation,
Convert the CIE LAB color space into an XYZ color indicator;
Compensating the saturation value of the image to be displayed by applying the chromaticity coordinates of the white point and a predetermined display reflectance in the converted XYZ color display system,
Convert the XYZ color indicator into a CIE LAB color space,
In the converted CIE LAB color space, when there is a color of an image out of the preset color range, the same color (HUE) at the position closest to the color of the image out of the preset color range among colors within the preset color range (HUE) And detecting a color having a value of) and mapping a color of an image which is out of the preset color range with the detected color.

KR1020130006517A 2013-01-21 2013-01-21 Method and apparatus for transformation of display image according to ambient light KR102030475B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020130006517A KR102030475B1 (en) 2013-01-21 2013-01-21 Method and apparatus for transformation of display image according to ambient light

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020130006517A KR102030475B1 (en) 2013-01-21 2013-01-21 Method and apparatus for transformation of display image according to ambient light

Publications (2)

Publication Number Publication Date
KR20140094161A KR20140094161A (en) 2014-07-30
KR102030475B1 true KR102030475B1 (en) 2019-11-08

Family

ID=51739990

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020130006517A KR102030475B1 (en) 2013-01-21 2013-01-21 Method and apparatus for transformation of display image according to ambient light

Country Status (1)

Country Link
KR (1) KR102030475B1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9934730B2 (en) 2014-11-19 2018-04-03 Samsung Electronics Co., Ltd. Computing device and image processing method thereof
KR102379774B1 (en) * 2015-08-31 2022-03-28 엘지디스플레이 주식회사 Image processing method, image processing circuit and display device using the same
WO2018207984A1 (en) 2017-05-12 2018-11-15 Samsung Electronics Co., Ltd. Electronic apparatus and method for displaying a content screen on the electronic apparatus thereof
CN108881758B (en) * 2018-05-29 2020-11-13 联发科技股份有限公司 Electronic equipment and method for processing image thereof
KR102574120B1 (en) * 2018-07-11 2023-09-04 삼성전자주식회사 Display apparatus and control method thereof
KR20210011162A (en) 2019-07-22 2021-02-01 삼성전자주식회사 Display apparatus and control method thereof
KR102276575B1 (en) * 2021-01-22 2021-07-14 삼익전자공업 주식회사 Intelligent Eelectronic Display Board That Increases Visibility By Responding To External Environmental Factors In Real Time

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005117612A (en) 2003-09-18 2005-04-28 Fuji Photo Film Co Ltd Image processing method and apparatus

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100735283B1 (en) * 2005-09-29 2007-07-03 삼성전자주식회사 Method for compensating a picture
KR100843089B1 (en) * 2006-10-17 2008-07-02 삼성전자주식회사 Apparatus and method for improving visibility for images
KR101464742B1 (en) * 2008-01-07 2014-11-25 삼성전자주식회사 Apparatus and method for providing visibility enhancement in portable terminal

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005117612A (en) 2003-09-18 2005-04-28 Fuji Photo Film Co Ltd Image processing method and apparatus

Also Published As

Publication number Publication date
KR20140094161A (en) 2014-07-30

Similar Documents

Publication Publication Date Title
KR102030475B1 (en) Method and apparatus for transformation of display image according to ambient light
US10419520B2 (en) Method of sharing electronic document and devices for the same
KR102051418B1 (en) User interface controlling device and method for selecting object in image and image input device
KR102158098B1 (en) Method and apparatus for image layout using image recognition
KR102052153B1 (en) Mobile terminal for controlling a hearing aid and method therefor
EP2790391B1 (en) Method and apparatus for displaying screen of portable terminal device
KR20190017303A (en) Mobile terminal
AU2014287980B2 (en) Portable device for providing combined UI component and method of controlling the same
KR20140126140A (en) Mobile apparatus providing with changed-shortcut icon responding to status of mobile apparatus and control method thereof
WO2020253295A1 (en) Control method and apparatus, and terminal device
KR20150100394A (en) Image displaying method and apparatus
WO2022028241A1 (en) Preview cover generation method and electronic device
KR20160127606A (en) Mobile terminal and the control method thereof
CN105681894A (en) Device and method for displaying video file
KR102226817B1 (en) Method for reproducing contents and an electronic device thereof
CN113269877B (en) Method and electronic equipment for acquiring room layout plan
US20140195990A1 (en) Mobile device system providing hybrid widget and associated control
KR20140068585A (en) Method and apparatus for distinction of finger touch and pen touch on touch screen
KR20140123325A (en) Method and apparatus for filling color of image
KR20140049324A (en) Method and apparatus for contents display according to handwriting
KR102069228B1 (en) Method and apparatus for filling color of image
WO2015027950A1 (en) Stereophonic sound recording method, apparatus, and terminal
KR20190027598A (en) Mobile terminal
KR102081659B1 (en) Method and apparatus for taking images for applying visual effect
KR102278676B1 (en) Method and apparatus for displaying user interface

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
AMND Amendment
E601 Decision to refuse application
AMND Amendment
X701 Decision to grant (after re-examination)
GRNT Written decision to grant