JP2011166491A - Method, apparatus and program for adjustment of image quality in display section using camera for mobile device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To adjust an image quality of a display section without increasing the size or costs and while suppressing an increase of power consumption, by effectively utilizing hardware resources of packaged various sensors or the like. <P>SOLUTION: In a terminal device having a camera for capturing an image, an illuminance sensor for measuring the ambient illuminance and a display section, a first image quality adjustment is performed to adjust display on the display section based on the illuminance measured by the illuminance sensor, and a second image quality adjustment is performed to adjust the display on the display section based on color temperature information of a photographing object, which is acquired when the camera performs auto white balance control. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to image quality adjustment of a display unit in a terminal device represented by a mobile phone.

  In recent years, the memory capacity of mobile phones has increased, and there are more and more usage scenes where you can carry various data and watch while moving. It is expected that users will use mobile phones in various environments more than ever before, and the demand for improving LCD (Liquid Crystal Display) image quality will increase further. However, it is time-consuming for the user to adjust the LCD image quality depending on the usage environment, and it may be difficult for some users to make adjustments using a mobile phone that has complicated functions due to its many functions. Therefore, mobile phones are required to make various adjustments automatically according to the usage environment without stress on the user.

  On the other hand, in mobile phones, mounting of a front camera for a TV phone and an illuminance sensor for reducing the power of the LCD has become almost standard.

  Therefore, in order to perform various adjustments as described above, there is a technique for adjusting the brightness of the LCD using an illuminance sensor so as to save power so as not to hinder the user. For example (see Patent Document 1). The brightness adjustment of the LCD can be realized by using the illuminance sensor as described above.

JP 2009-109876 A

  However, it is not always possible to provide a satisfactory image quality for the user only by adjusting the luminance. This point will be described by comparing with a home TV.

  Recently, many home TVs have been released that offer various functions and the user can set the image quality so that higher image quality can be viewed. Optimum image quality is achieved according to various sensors and time. Some automatically adjust the state. However, it is difficult to add hardware such as sensors only for image quality adjustment for adjustment in mobile products with extremely high size and cost constraints. Furthermore, since the addition of hardware is concerned about the influence on power consumption (usage time) that is important for mobile products, it is difficult to adjust the LCD image quality in the same manner as for home TVs.

  However, even without adding hardware only for image quality adjustment, mobile phones, among other mobile products, are already equipped with various sensors and functions as standard for other purposes.

  Therefore, the present invention provides a camera for a mobile phone that can effectively use hardware resources such as various sensors mounted thereon, and can perform control while suppressing an increase in power consumption without an increase in size or cost. It is an object of the present invention to provide an image quality adjustment method, an apparatus thereof, and a program thereof used for a display unit.

  According to a first aspect of the present invention, in a terminal device having a camera for imaging, an illuminance sensor for measuring ambient illuminance, and a display unit, the display unit based on the illuminance measured by the illuminance sensor A second image quality adjustment for adjusting the display of the display unit based on the color temperature information of the object to be captured that is acquired when the camera performs auto white balance adjustment. There is provided a terminal device characterized by:

  According to a second aspect of the present invention, in an image quality adjustment method performed by a terminal that includes a camera for imaging, an illuminance sensor that measures illuminance around, and a display unit, based on the illuminance measured by the illuminance sensor. The first image quality adjustment for adjusting the display on the display unit is performed, and the display on the display unit is adjusted based on the color temperature information of the shooting target acquired when the camera performs auto white balance adjustment. An image quality adjustment method characterized by performing image quality adjustment 2 is provided.

  According to a third aspect of the present invention, in an image quality adjustment program incorporated in a terminal device having a camera for imaging, an illuminance sensor for measuring ambient illuminance, and a display unit, the illuminance measured by the illuminance sensor The first image quality adjustment for adjusting the display on the display unit is performed based on the image, and the display on the display unit is adjusted based on the color temperature information of the shooting target acquired when the camera performs auto white balance adjustment. An image quality adjustment program is provided that causes a computer to function as a terminal device that performs the second image quality adjustment.

  According to the present invention, it is possible to effectively use hardware resources such as various mounted sensors and not activate various sensors for more than necessary time, thereby increasing power consumption without increasing size and cost. It is possible to adjust the image quality while suppressing it.

It is a block diagram showing the basic composition of the embodiment of the present invention. It is a figure showing the example of mounting of the basic composition of the embodiment of the present invention. It is a flowchart (1/2) showing the basic operation | movement of embodiment of this invention. It is a flowchart (2/2) showing the basic operation | movement of embodiment of this invention. It is a sequence diagram showing the basic operation | movement of embodiment of this invention. It is a figure showing the example of mounting of the basic composition of the embodiment of the present invention. It is a figure showing the example of mounting of the basic composition of the embodiment of the present invention.

  First, an outline of an embodiment of the present invention will be described. The embodiment of the present invention takes advantage of the characteristics of a mobile phone incorporating various devices and functions such as an illuminance sensor and a camera. In this embodiment, there is no hardware change, only the system change is performed, and the LCD image quality is adjusted according to the user's surrounding environment detected by each sensor, thereby providing high power LCD image quality to the user. it can.

  In recent mobile phones, mounting of a front camera for a TV phone and an illuminance sensor for reducing the power of the LCD has become almost standard. Then, the brightness of the LCD is adjusted using an illuminance sensor to save power so as not to hinder the user. The operation of the front camera is linked to the detection value of this illuminance sensor, and the function of the camera's auto white balance (hereinafter referred to as “AWB”) is used to identify the color temperature of the user's surrounding environment and adjust the image quality of the LCD. By applying the feedback, the LCD image quality adjustment that the user feels higher in image quality than the environment is realized.

  The above is the outline of the embodiment of the present invention.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 1, a mobile phone 100 according to this embodiment includes a main camera 101, a sub LCD 102, a main LCD 103, an illuminance sensor 104, a sub camera 105, and an arithmetic processing unit 106.
FIG. 2 is a configuration diagram when it is assumed that the present embodiment is mounted on a clamshell type mobile phone (sometimes referred to as a “folding type”). As will be described later, the present embodiment is not limited to a clamshell type mobile phone. FIG. 2 shows an example only.

  The main camera 101 is a camera that is mainly used. In recent years, the main camera 101 is usually a high-pixel camera of several megapixels or more. While it has high image quality performance, it consumes a large amount of current.

  The sub LCD 102 is an auxiliary display device that is mounted in a shape that closes the housing when not in use, such as a clamshell type, and generally has a small size and low resolution. ing.

  The main LCD 103 is a display device used when using various functions of a normal mobile phone. Recently, the main LCD 103 has a screen resolution of about 640 × 480 pixels. In general, a screen resolution of 640 × 480 pixels may be referred to as VGA (Video Graphics Array).

  The illuminance sensor 104 is a sensor for measuring illuminance. The illuminance sensor 104 measures the illuminance of the user environment on the main LCD surface side, and in order to save power of the main LCD, the brightness of the LCD is low in a dark environment and the brightness of the LCD is high in a bright environment. Used for LCD brightness adjustment.

  The sub camera 105 is a camera that is used for a normal videophone or self-portrait. Nowadays, the number of pixels of the VGA class is common.

  FIG. 1 is a simple block diagram of the interior of the mobile phone, in which each device (main camera 101, sub LCD 102, main LCD 103, illuminance sensor 104, and sub camera 105) is controlled by the arithmetic processing unit 106. Show. The arithmetic processing unit 106 is a processor for arithmetic processing represented by a CPU (Central Processing Unit). The processor operates as the arithmetic processing unit 106 in cooperation with software.

  Each of these devices is composed of a number of members, and there are some complicated parts such as a case where each device is separately mounted, but here, for simplicity, the devices are grouped as one device.

  Note that the camera devices (the main camera 101 and the sub camera 105) have an AWB function. The camera device is assumed to have a function of outputting color temperature information of the shooting environment adjusted at the time of AWB adjustment.

  Next, the operation of this embodiment will be described in detail using the flowcharts of FIGS. 3-1 and 3-2 and the sequence diagram of FIG.

  The flowcharts of FIGS. 3-1 and 3-2 and the sequence diagram of FIG. 4 show the arithmetic processing unit 106, the main LCD 103, the illuminance sensor 104, and the sub camera 105 for the surrounding environment used by the user and the state of the mobile phone. Each operation is shown.

  First, with reference to FIG. 3A, a case will be described in which a cellular phone is opened or a key is pressed to wake up at the time of start-up or for incoming calls or other purposes.

  First, each device starts or wakes up (step S11), and the illuminance sensor 104 measures the illuminance of the environment on the main LCD 103 side (step S12).

  Next, the arithmetic processing unit 106 adjusts the luminance of the main LCD 103 according to the measurement result of the illuminance sensor 104 (step S13). Up to this point, it is the main LCD 103 control of the current general mobile phone.

  In this embodiment, unlike a general mobile phone, the sub camera 105 is also activated simultaneously in step S11. Then, the sub camera 105 acquires an image on the main LCD 103 side and performs AWB adjustment (step S14). At this time, the inside of the device of the sub camera 105 holds the color temperature information of the photographing target that is the adjustment target when the AWB adjustment is performed (step S15).

  The arithmetic processing unit 106 reads out the color temperature information held by the sub camera 105 (step S16).

  On the other hand, since the sub-camera 105 is generally a device that consumes a large amount of current, if it is activated, it greatly affects the usage time of the mobile phone. Therefore, it is detected from the illuminance sensor 104 that there has been a significant change in the usage environment, and the camera 105 is activated only when the white balance (hereinafter referred to as “WB”) of the main LCD 103 needs to be readjusted. It is desirable to acquire temperature information. Therefore, in this embodiment, after the arithmetic processing unit 106 reads the color temperature information, the sub camera 105 is turned off (step S17).

  Then, the arithmetic processing unit 106 determines the color temperature (main peripheral light source) of the main LCD 103 side environment, and performs WB adjustment of the main LCD 103 so that it is easy for the user to see according to the peripheral environment (step S18). Here, the fineness of the WB adjustment can be arbitrarily set. However, if the WB adjustment is set too fine, the change of the main LCD 103 is noticeable and may adversely affect the user. Therefore, there are several types of WB adjustment values, such as incandescent lamps with low color temperature, under fluorescent lamps, outdoors, sunny outdoors, or cloudy, with different WB adjustment values. It is considered that the LCD 103 should be adjusted.

  Next, the operation after step S18 will be described with reference to FIG.

  The features of the operation after step S18 are as follows. This point will be described. Normally, when the environment changes and WB adjustment of the main LCD 103 is necessary, it is considered that a large change in luminance is likely to occur. Therefore, the measurement result of the illuminance sensor 104 is used as a trigger as a detection method of the environmental change. That is, the feature of this embodiment is that the measurement result of the illuminance sensor 104 and the activation of the camera 105 are linked.

  After performing the WB adjustment of the main LCD 103 in step S18, it is determined whether or not the illuminance change in the surrounding environment is large (step S19). Although it is a criterion for determining whether or not it is large, it is possible to adopt an arbitrary criterion in consideration of the capability of each device of the mobile phone.

  As a result of the determination in step S19, it is assumed that a small illuminance change in the surrounding environment is detected (No in step S19).

  In this case, since there is no large change in illuminance as an environment, the light source (WB) in the surrounding environment is not changed, and the WB adjustment of the main LCD 103 is unnecessary. Therefore, only the luminance adjustment of the main LCD 103 based on the measurement result of the illuminance sensor 104 is performed (step S20).

  Next, a description will be given of when the illuminance changes in a large surrounding environment. When the amount of change in illuminance that is considered to have changed the light source (WB) in the surrounding environment is stored as data on the arithmetic processing unit 106 side and exceeds the threshold value (change amount) (Yes in step S19), the sub camera 105 Is restarted (step S21). Thereafter, the AWB adjustment is performed in the same manner as in steps S14 to S17, and the arithmetic processing unit 106 performs the WB adjustment of the LCD 103 according to the color temperature by acquiring the color temperature information of the AWB adjustment again (steps S22 to 25). ). Thereafter, the sub camera 105 is turned OFF in view of the large current consumption of the sub camera 105 as in step S18 (step S26).

  In FIGS. 3A and 3B, the illuminance measurement by the illuminance sensor 104 is continued. In order to explain this point, FIG. 4 shows the above operation as a sequence diagram. The operation of the part denoted by (1) in FIG. 4 is an operation corresponding to step S11 to step S18 shown in FIG. The operation indicated by (2) is the operation corresponding to NO in step S19 shown in FIG. Moreover, the operation | movement of the part described as (3) is operation | movement equivalent to YES in step S19 shown to FIG. 3-2, and step S21-step S26.

  By performing the operation as described above, this embodiment can adjust the LCD suitable for the surrounding environment while suppressing an increase in power consumption due to a change in the surrounding environment, and can provide an optimum image quality to the user.

  There are digital still cameras and camcorders as products equipped with a camera and LCD. However, since the shooting direction and the LCD direction are opposite, the same usage environment as the user's usage environment (light source determination in the environment where the user is present) is not necessarily required. It's not possible. In view of this point, the feature of this embodiment is that the characteristics of a mobile phone in which an illuminance sensor and a camera are mounted on the LCD side are used.

  Next, another embodiment of the present invention will be described.

  First, the description so far has been based on the premise of the WB adjustment of the main LCD, but a similar function can be realized by a combination of the sub LCD and the main camera. In the present embodiment, an illuminance sensor and a camera are mounted, and the mounting direction of these devices takes advantage of the characteristics of the mobile phone that is optimal for the present invention. However, the present embodiment can be applied to any product having a similar hardware configuration, not limited to a mobile phone. That is, in the above-described embodiment, a mobile phone has been described as an example, but this embodiment is applicable to many terminals such as a terminal conforming to PHS (Personal Handy-phone System) and a portable game machine equipped with a camera. Can be implemented. Further, the present invention can be applied to a display unit of a system other than the LCD. Furthermore, when applied to a mobile phone, it may be applied not only to the clamshell type shown in FIG. 2, but also to the straight type shown in FIG. 5 and the slide type shown in FIG.

  Also, some recent mobile phone cameras have a function of identifying a shooting scene. If the number of scenes to be identified increases in the future, it is considered possible to perform LCD image quality adjustment optimal for the scene based on the scene identification result as well as the camera AWB information. For example, it is possible to identify that there are two or more people on the main LCD side, detect that you want to see the LCD screen together, and turn off automatically if the peep prevention function is on. It is done.

  The embodiment of the present invention described above has many effects as described below.

  The first effect is that it is possible to provide an LCD image quality that is optimally adjusted to the user simply by changing the system with an existing hardware configuration.

  The reason for this is that the WB adjustment of the LCD is automatically performed in accordance with the user's usage environment, taking advantage of the features of a mobile phone equipped with an illuminance sensor, a camera and the like.

  The second effect is that an optimally adjusted LCD image quality can be provided while suppressing an increase in power consumption.

  The reason is that a change in the user's usage environment is detected by the illuminance sensor, and only when there is a change in the usage environment (a change in the light source), the camera is activated and the light source determination of the usage environment is performed from AWB adjustment.

  The image quality adjustment apparatus for a display unit using a camera for a mobile phone according to an embodiment of the present invention can be realized by hardware, but the image quality adjustment apparatus for a display unit using the camera for a mobile phone is used as a computer. It is also possible for the computer to function as a computer program by reading it from a computer-readable recording medium and executing it.

  In addition, the image quality adjustment method for the display unit using the mobile phone camera according to the embodiment of the present invention can be realized by hardware, but the computer can read the program for causing the computer to execute the method. It can also be realized by reading from a recording medium and executing it.

  Moreover, although the above-described embodiment is a preferred embodiment of the present invention, the scope of the present invention is not limited only to the above-described embodiment, and various modifications are made without departing from the gist of the present invention. Implementation in the form is possible.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Supplementary Note 1) In an image quality adjustment method performed by a terminal having a camera for imaging, an illuminance sensor that measures ambient illuminance, and a display unit,
Based on illuminance measured by the illuminance sensor, a first image quality adjustment is performed to adjust the display of the display unit, and based on color temperature information of a subject to be acquired acquired when the camera performs auto white balance adjustment. A second image quality adjustment method for adjusting the display of the display unit.

(Appendix 2) In the image quality adjustment method described in appendix 1,
In the first image quality adjustment, the brightness of the display unit is adjusted, and in the second image quality adjustment, the white balance of the display unit is adjusted.

(Supplementary Note 3) In the image quality adjustment method according to Supplementary Note 1 or 2,
When the camera is turned off after performing the first adjustment and the second adjustment at the time of start-up or wake-up, and the illuminance sensor does not detect an illuminance change exceeding a threshold value, the first without adjusting the camera An image quality adjustment method characterized by performing only one image quality adjustment.

(Supplementary note 4) In the image quality adjustment method according to any one of supplementary notes 1 to 3,
The camera is turned off after the first adjustment and the second adjustment at the start-up or wake-up, and when the illuminance sensor detects an illuminance change exceeding a threshold, the camera is restarted and restarted. An image quality adjustment method, wherein the second image quality adjustment is performed again based on the color temperature information of the photographing target acquired when the camera performs auto white balance adjustment again.

(Supplementary note 5) In the image quality adjustment method according to any one of supplementary notes 1 to 4,
The image quality adjustment method, wherein the camera, the illuminance sensor, and the display unit are arranged on the same surface of the terminal device.

(Additional remark 6) In the image quality adjustment program built in the terminal device which has the camera for imaging, the illumination intensity sensor which measures the illumination intensity around, and a display part,
Based on illuminance measured by the illuminance sensor, a first image quality adjustment is performed to adjust the display of the display unit, and based on color temperature information of a subject to be acquired acquired when the camera performs auto white balance adjustment. An image quality adjustment program that causes a computer to function as a terminal device that performs second image quality adjustment for adjusting display on the display unit.

(Appendix 7) In the image quality adjustment program described in appendix 6,
What is adjusted in the first image quality adjustment is brightness of the display unit, and what is adjusted in the second image quality adjustment is white balance of the display unit.

(Appendix 8) In the image quality adjustment program described in appendix 6 or 7,
When the camera is turned off after performing the first adjustment and the second adjustment at the time of start-up or wake-up, and the illuminance sensor does not detect an illuminance change exceeding a threshold value, the first without adjusting the camera An image quality adjustment program characterized by performing only one image quality adjustment.

(Supplementary note 9) In the image quality adjustment program according to any one of supplementary notes 6 to 8,
The camera is turned off after the first adjustment and the second adjustment at the start-up or wake-up, and when the illuminance sensor detects an illuminance change exceeding a threshold, the camera is restarted and restarted. An image quality adjustment program for performing the second image quality adjustment again based on the color temperature information of the photographing target acquired when the camera performs auto white balance adjustment again.

(Supplementary Note 10) In the image quality adjustment program according to any one of Supplementary Notes 6 to 9,
The image quality adjustment program, wherein the camera, the illuminance sensor, and the display unit are arranged on the same surface of the terminal device.

100 Mobile phone 101 Main camera 102 Sub LCD
103 Main LCD
104 Illuminance sensor 105 Sub camera 106 Arithmetic processing unit

Claims (7)

In a terminal device having a camera for imaging, an illuminance sensor for measuring ambient illuminance, and a display unit,
Based on illuminance measured by the illuminance sensor, a first image quality adjustment is performed to adjust the display of the display unit, and based on color temperature information of a subject to be acquired acquired when the camera performs auto white balance adjustment. A terminal device that performs a second image quality adjustment for adjusting the display of the display unit. The terminal device according to claim 1,
The terminal device characterized in that the brightness of the display unit is adjusted in the first image quality adjustment, and the white balance of the display unit is adjusted in the second image quality adjustment. The terminal device according to claim 1 or 2,
When the camera is turned off after performing the first adjustment and the second adjustment at the time of start-up or wake-up, and the illuminance sensor does not detect an illuminance change exceeding a threshold value, the first without adjusting the camera A terminal device that performs only one image quality adjustment. The terminal device according to any one of claims 1 to 3,
The camera is turned off after the first adjustment and the second adjustment at the start-up or wake-up, and when the illuminance sensor detects an illuminance change exceeding a threshold, the camera is restarted and restarted. A terminal device, wherein the second image quality adjustment is performed again based on the color temperature information of the photographing target acquired when the camera performs automatic white balance adjustment again. The terminal device according to any one of claims 1 to 4,
The said camera, the said illumination intensity sensor, and the said display part are arrange | positioned on the same surface of the said terminal device, The terminal device characterized by the above-mentioned. In an image quality adjustment method performed by a terminal having a camera for imaging, an illuminance sensor that measures ambient illuminance, and a display unit,
Based on illuminance measured by the illuminance sensor, a first image quality adjustment is performed to adjust the display of the display unit, and based on color temperature information of a subject to be acquired acquired when the camera performs auto white balance adjustment. A second image quality adjustment method for adjusting the display of the display unit. In an image quality adjustment program incorporated in a terminal device having a camera for imaging, an illuminance sensor that measures the illuminance of the surroundings, and a display unit,
Based on illuminance measured by the illuminance sensor, a first image quality adjustment is performed to adjust the display of the display unit, and based on color temperature information of a subject to be acquired acquired when the camera performs auto white balance adjustment. An image quality adjustment program that causes a computer to function as a terminal device that performs second image quality adjustment for adjusting display on the display unit.

Images