JP2018042155A - Display apparatus and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display apparatus capable of suppressing sudden luminance change while alleviating difficulty of viewing when the external light illuminance applied to the display apparatus changes greatly, such as when a user is observing and a display control method for the same.SOLUTION: The display apparatus includes: external light illuminance detection means that detects illuminance of external light irradiated on a display apparatus; acquisition means that acquires a target luminance value corresponding to the detected external light illuminance; and a line of sight detection means that detects a direction of a user's line of sight. On the basis of the detection result of the line of sight detection means, the display apparatus controls an amount of change in the luminance of the display apparatus.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device that displays an image and a control method thereof.

  2. Description of the Related Art Conventionally, many digital cameras that record and display an electronic image photoelectrically converted by an image sensor have a display device for displaying information necessary for shooting or displaying a recorded image. Yes. A TFT liquid crystal display device is generally used as a device that is actually used in a display device. In addition, an organic EL display device may be used. In the case of a digital camera, the brightness of the environment in which it is used varies, such as when it is outdoors in a clear sky or when it is in a dark room. Therefore, the brightness of the image displayed on the display device is automatically adjusted to the optimum brightness according to the brightness of the ambient light.

  Patent Documents 1 and 2 show examples in which the brightness of an image or the like displayed on a display device is automatically adjusted according to the brightness of ambient light. The electronic camera described in Patent Document 1 adjusts the brightness of a backlight of a liquid crystal panel that displays an image by obtaining brightness information of ambient light, that is, illuminance of external light illuminating the camera, from video information obtained by an image sensor. Note that, in this Patent Document 1, a device for lowering the brightness of a backlight is disclosed in order to secure power to the charging side when a strobe provided in a camera is being charged.

  In addition, the camera described in Patent Document 2 adjusts the luminance of the backlight of the liquid crystal displaying the image in consideration of not only the illuminance of the ambient light but also the brightness information of the captured image obtained through the photographing lens. A method is disclosed. In any of the cameras disclosed in Patent Documents 1 and 2, the luminance of the liquid crystal display device can be changed in accordance with the change in the brightness of the external light illuminance, and can be adjusted to an easily viewable luminance.

JP-A-8-242398 JP 2010-147862 A

  However, the conventional techniques disclosed in the above-mentioned patent documents have the following problems. For example, when the illuminance of the external light irradiated to the camera changes greatly due to the sun being blocked or not blocked by the photographer when used outdoors, the brightness of the liquid crystal display device suddenly changes according to the change of the illuminance of the external light. Sometimes it becomes brighter and darker. As described above, when the luminance adjustment is performed according to the change in the illuminance of the external light, the sudden change in brightness caused by the adjustment may make it difficult for the user of the camera to see.

  Therefore, an object of the present invention is a state in which the illuminance of outside light applied to the display device changes greatly, and when the user is observing, a sudden change in luminance is suppressed to reduce the difficulty of viewing. It is to provide a display control device and a control method thereof.

In order to achieve the above object, the present invention provides a display control device for controlling a display device for displaying an image, the illuminance detection means for detecting the illuminance of external light applied to the display device, and the detected outside Luminance acquisition means for acquiring a target luminance value corresponding to the light illuminance, and control means for controlling the luminance of the display device so as to be a luminance setting value set based on the acquired target luminance value;
Visual line detection means for detecting the direction of the visual line of the user, and the amount of change in luminance of the display device is controlled based on the detection result of the visual line detection means.

  According to the present invention, by changing the brightness control method of the display device according to the user's line-of-sight detection result, it is possible to reduce sharpness by suppressing a sudden brightness change when the user is observing. I can do it. In addition, when the line of sight is deviated, the luminance is changed without limitation on the amount of change in luminance. Therefore, when the line of sight is returned to the display device, it is displayed with a luminance that is easy to see and provides a display device with good visibility. I can do it.

1 is a block diagram of a digital single-lens reflex camera according to an embodiment of the present invention. External view of camera body 1 viewed from the back The system block diagram of the gaze detection means which concerns on embodiment of this invention The flowchart in the 1st Embodiment of this invention Flowchart in the second embodiment of the present invention

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Example 1]
FIG. 1 is a block diagram showing a configuration of an electric circuit of a digital camera equipped with a display device according to a first embodiment of the present invention. The digital camera includes a camera body 1 and a lens 50, and the lens 50 is configured to be detachable from the camera body 1. In FIG. 1, a power supply circuit unit 2 of the camera body 1 receives power supply from a battery 3 and generates a stabilized power supply to enable each circuit component described below to operate.

  The control means 4 performs overall control of the camera body 1. The control means 4 outputs a control signal for power supply to each circuit component to the power supply circuit unit 2 in accordance with various operation modes. Further, a release switch 5 and an operation member 6 for instructing a shooting preparation operation and a shooting start are connected to the control means 4. The operation member 6 is a switch for setting various camera operations. The subject light transmitted through the lens 50 passes through the optical filter 7 in which the optical low-pass filter and the infrared cut filter are integrated, and is exposed to the image sensor 8.

  The image sensor 8 is composed of a two-dimensional area sensor such as a CCD or a CMOS that converts an optical image into an electric signal. The output of the image sensor 8 is subjected to noise reduction and gain adjustment by the CDS / AGC 9, converted from an analog signal to a digital signal by the A / D conversion circuit 10, and input to the image processing unit 11.

  The image processing unit 11 performs image processing such as color adjustment, contrast adjustment, and edge enhancement (contour enhancement) on the image information, and compresses or decompresses the image information in order to store and read the image information in the storage unit 12. The image data conversion process is performed. The storage unit 12 includes a detachable semiconductor memory, a magnetic disk, an optical disk, etc., and stores captured image information.

  Further, the image processing unit 11 calculates the luminance of the subject and notifies the control unit 4 of the luminance. The control unit 4 controls the shutter (not shown) and the aperture (not shown) of the lens 50 based on the acquired luminance information of the subject, and controls the exposure amount to the image sensor 8. Further, the image processing unit 11 drives a timing generator (TG / VDR) 13 to generate a clock signal and the like to be supplied to the image sensor 8. A primary memory 14 is connected to the control means 4 and the image processing unit 11 and is used for storing a program to be provided, temporarily storing a photographed image, and as a work area.

  The image processing unit 11 outputs the display image information to the display unit 15, and the captured image is displayed on the monitor. The display means 15 is composed of a liquid crystal device or the like, and in the case of the liquid crystal device, the inside is divided into a liquid crystal panel 15a and a backlight 15b. The display brightness of the liquid crystal device is realized by changing the luminance of the backlight 15b.

  The illuminance sensor 16 outputs a signal corresponding to the illuminance of outside light irradiated on the back of the camera by a photoelectric conversion element such as SPC (silicone photo cell). The line-of-sight detection means 17 is a means for detecting the direction of the user's line of sight, and includes, for example, an infrared light emitting unit and a light receiving unit. The control unit 4 controls the luminance of the backlight 15 a according to the output of the illuminance sensor 16 and the detection result of the line-of-sight detection unit 17.

  An attitude detecting means 18 is further connected to the control means 4. The posture detection means 18 is composed of an acceleration sensor or the like, detects the orientation or change of the camera body 1, and the control means 4 performs camera control according to the detection result.

  FIG. 2 is an external view of the camera body 1 viewed from the back. On the back surface of the camera body 1, an eyepiece 201 of an optical finder, a display unit 15, an illuminance sensor 16, and a line-of-sight detection unit 17 are arranged. Also, a release switch 5 and various operation members 6 are arranged.

  The shooting mode dial 6a is an operation member that can be selected to select various shooting modes such as program auto, shutter priority, aperture priority, and manual by rotating, and the menu button 6b is used to display a setting menu such as camera functions. Used for. The reproduction button 6c is an operation member used when the recorded image is reproduced on the display means 15, and the erasing button 6d is used when erasing the displayed recorded image. The cross button 6e can be input up and down or left and right on a menu display screen or the like, and selects various setting items of the camera. The power switch 6f is a switch for controlling on / off of the operation of the camera. When the power switch 6f is in the on state, the camera accepts various operations such as shooting by the user and image reproduction. When the power switch is in the off state, the camera is in the power saving mode, and operations other than the power switch are not accepted.

  The illuminance sensor 16 is desirably arranged on the same surface as the display unit 15 for the purpose of adjusting the luminance of the backlight 15b of the display unit 15. Further, when the illuminance sensor 16 is blocked by the operator's hand when operating the various operation members 6, the illuminance of the external light cannot be accurately measured. Therefore, it is desirable that the illuminance sensor 16 is arranged so as not to block outside light even when the operation member 6 is operated. Further, the line-of-sight detection means 17 is preferably arranged on the same surface as the display means 15 for the purpose of determining whether or not the user is viewing the display means 15.

  Further, if the line of sight to the line-of-sight detection means 17 is blocked by the operator's hand when operating the various operation members 6, the line of sight cannot be detected. Therefore, it is desirable that the line-of-sight detection means 17 is arranged so as not to block the line of sight even when the operation member 6 is operated.

  FIG. 3 is a system configuration diagram showing a configuration example of the line-of-sight detection means 17 in the present embodiment. The light source 301 is an infrared LED that emits light at a wavelength in the infrared region. In response to an instruction to start line-of-sight detection from the control unit 4, the light source control unit 302 controls turning on and off the light source 301 in accordance with the line-of-sight detection operation. The imaging control unit 303 controls the timing at which the eyeball imaging unit 304 captures an image during the line-of-sight detection process. When notified from the light source control unit 302 that the light source 301 is turned on, the imaging control unit 303 transmits an imaging instruction signal to the eyeball imaging unit 304.

  The face detection unit 305 detects a region in which the face is reflected from the image captured by the eyeball imaging unit using a known template matching or the like in order to specify the region in which the photographer's eyes are reflected. The Purkinje image detection unit 306 detects a region in which an eye is reflected from the face regions detected by the face detection unit 305, and detects a cornea reflection image (hereinafter referred to as a Purkinje image) and a pupil of the light source 301 from the detected region. The eye and pupil are detected by, for example, known template matching between the eye and pupil template image and the face region image.

  The luminance of the region where the Purkinje image of the light source 301 is reflected is higher than that of the surrounding area. Therefore, a region that substantially matches the contour shape of the light-emitting surface of the light source 301 in the eye region and has a higher luminance than the surroundings is detected as a Purkinje image. Then, the Purkinje image detection unit 306 notifies the eye gaze detection unit 307 of the center of gravity of the pupil and the center of gravity of the Purkinje image.

  The line-of-sight detection unit 307 detects the line-of-sight direction of the photographer based on the center of gravity of the pupil and the center of gravity of the Purkinje image. The position of the Purkinje image of the light source is constant regardless of the line-of-sight direction because the surface of the cornea is substantially spherical.

  On the other hand, the center of gravity of the pupil moves in accordance with the line of sight of the photographer. Therefore, the line-of-sight detection unit 307 can detect the direction of the user's line of sight by obtaining the relative position of the center of the pupil relative to the center of gravity of the Purkinje image.

  The gaze detection unit 307 notifies the gaze point detection unit 309 of the user's gaze direction. The distance estimation unit 308 detects organs such as eyes and nose from the face area obtained by the face detection unit 305, and obtains an inter-organ distance. Further, the distance estimation unit 308 calculates the user distance between the user and the display unit 15 according to the calculated inter-organ distance, and notifies the gaze point detection unit 309.

  The gaze point detection unit 309 determines whether or not the user's gaze direction is the display unit 15 based on the gaze direction detection result of the gaze detection unit 307 and the user distance output result from the distance estimation unit 308, or the subject direction Or not, and notifies the control unit 4 of the determination result.

  In the present embodiment, the above-described detection method using a cornea reflection image has been described as an example of the line-of-sight detection means 17, but the present invention is not limited to this. An image on the back of the camera may be acquired using an image sensor different from the image sensor that acquires the image of the main subject, and the line of sight may be detected from the acquired image information, or other methods may be used.

  FIG. 4 is a flowchart of the brightness changing process of the backlight 15b of the display means of the digital camera according to the present embodiment. When the user operates the operation member 6, various setting screens of the camera, captured images, or live view images are displayed on the display unit 15 according to the operation content. When the display unit 15 is in the display state of the item, the control sequence of the luminance of the backlight 15b is executed.

  (S401) The current brightness setting value of the backlight 15b is set to B0.

  (S402) The external light illuminance is acquired from the illuminance sensor 16, and the acquired external light illuminance is set to L1.

  (S403) The control means 4 calculates the target brightness setting value B1 of the backlight 15b from the acquired external light illuminance L1.

  (S404) The control means 4 compares the current luminance setting value B0 with the calculated target luminance setting value B1, and if the luminance difference is greater than or equal to the predetermined luminance threshold Bx, the control unit 4 proceeds to S405. If the luminance difference is less than the luminance threshold Bx, the luminance of the backlight 15b is not changed, and the process proceeds to S412.

  Here, the brightness threshold value Bx is a change amount that does not affect the appearance of the display unit 15 such as variations in detection of the illuminance sensor and unevenness of external light, and is a threshold value for avoiding repeated minute brightness changes. In the case of a luminance change amount smaller than the luminance threshold Bx, the luminance change is not performed. The luminance threshold value Bx may be a fixed value, or may be calculated as a ratio from a value corresponding to the current luminance value. However, when the brightness threshold is set to a small value, the brightness is changed even with a slight change in the illuminance of outside light. When the brightness threshold is set to a large value, it is difficult to change the brightness.

  (S405) The control means 4 compares the current brightness setting value B0 with the calculated target brightness setting value B1, and if the brightness difference is less than the predetermined brightness change limit value Blim, changes to the target brightness setting value B1. Even if it is determined that the user does not feel uncomfortable, the process proceeds to S406. If the luminance difference is equal to or larger than the luminance change limit value Blim, it is determined that the user feels uncomfortable when the target luminance setting value B1 is changed, and the amount of luminance change is limited in step S408. Migrate to

  Here, the brightness change limit value Blim is a change amount that the user does not feel uncomfortable when the brightness of the backlight is changed, and may be a fixed value or a ratio from a value according to the current brightness value. It may be calculated.

  (S406) The control means 4 determines that the user does not feel uncomfortable even when the target brightness setting value B1 is changed, and changes the brightness setting value Bs to the target brightness setting value B1.

  (S407) The control means 4 changes the luminance of the backlight 15b based on the luminance setting value Bs.

  (S408) If it is determined from the detection result of the line-of-sight direction of the line-of-sight detection means 17 that the user's line of sight has deviated from the display means 15, the user becomes uncomfortable even if the target brightness setting value B1 is changed. Judge that you do not feel. The process proceeds to S406, and the luminance of the backlight 15b is changed with the target luminance setting value B1 as the luminance setting value Bs.

  (S409) The control means 4 determines whether the target brightness setting value B1 is brighter or darker than the current brightness setting value B0 in order to calculate a brightness setting value with a limited amount of change. It is determined whether or not the subtraction result between the current brightness setting value B0 and the target brightness setting value B1 takes a positive value. If the subtraction result takes a positive value, the process proceeds to S410. , The process proceeds to S411.

  (S410) The control means 4 changes the brightness setting value Bs of the backlight 15b to a value obtained by subtracting the brightness change limit value Blim from the current brightness setting value B0 in order to determine the brightness set value with the change amount limited.

  (S411) The control means 4 changes the brightness setting value Bs of the backlight 15b to a value obtained by adding the brightness change limit value Blim to the current brightness setting value B0 in order to determine the brightness setting value with the change amount limited.

  (S412) The control unit 4 determines whether to end or continue the display operation of the display unit 15, and when the display operation ends, the process proceeds to S413. If it is determined that the display operation is to be continued, the process proceeds to S414.

  (S413) The control means 4 turns off the backlight 15b, performs the termination process of the display means 15, and ends the luminance control sequence.

  (S414) The control means 4 waits for a predetermined time, returns to S401, and restarts the luminance control sequence of the display means.

  Accordingly, if the luminance difference between the current luminance setting value B0 and the target luminance setting value B1 is equal to or greater than the luminance change limit value Blim, the target corresponding to the ambient light illuminance L1 stepwise with the waiting time of S414. The brightness setting value B1 is changed.

  Here, the waiting time of S414 may be a fixed value, or when there is a concern that the time to change when the external light illuminance changes suddenly is too long, the time may be varied according to the luminance difference.

  As described above, according to the present embodiment, it is possible to reduce the difficulty of viewing by suppressing a sudden change in luminance when the user is observing, based on the user's line-of-sight detection result. In addition, when the line of sight is deviated, the luminance is changed without limitation on the amount of change in luminance. Therefore, when the line of sight is returned to the display device, it is displayed with a luminance that is easy to see and provides a display device with good visibility. It becomes possible.

[Example 2]
In the second embodiment, a method of changing the luminance of the backlight 15b will be described in which the detection result of the line-of-sight detection unit 17 is further utilized and combined with the posture detection unit 18. Note that the configuration (see FIG. 1) and appearance (see FIG. 2) of the electric circuit of the digital camera of the second embodiment are the same as those in the first embodiment, and the description thereof is omitted.

  Hereinafter, with reference to FIG. 5, a luminance change sequence according to the second embodiment of the present invention will be described.

  (S501) The current brightness setting value of the backlight 15b is set to B0.

  (S502) If the detection result by the posture detection means 18 is a fluctuation for a predetermined time or more and a predetermined range or less, it is determined that the posture of the camera body is the same, and the process proceeds to S503. That is, when the posture of the camera body 1 is changed, it is determined that the user is not looking at the display means, the luminance is not changed, and the posture of the camera body 1 is not changed. Start.

  (S503) The external light illuminance is acquired from the illuminance sensor 16, and the acquired external light illuminance is set to L1.

  (S504) When it is determined from the detection result of the line-of-sight direction of the line-of-sight detection unit 17 that the user's line of sight is facing the display unit 15, the process proceeds to S505. If it is determined that the user's line of sight has deviated from the display means 15, the process proceeds to S506.

  (S505) The control unit 4 calculates the target brightness setting value B1 of the backlight 15b from the acquired external light illuminance L1.

  (S506) If it is determined from the detection result of the line-of-sight direction of the line-of-sight detection means 17 that the user's line of sight is directed toward the subject, the process proceeds to S507 in order to correct the target luminance setting value according to the subject luminance. To do. If it is determined that the user's line of sight is not directed toward the subject, it is determined that the user does not feel uncomfortable even if the target brightness setting value B1 is changed, and the process proceeds to S510.

  (S507) The image processing unit 11 calculates the overall luminance of the subject from the image signal of the subject from the image sensor 8, and notifies the control means 4 of the luminance. The control unit 4 calculates a target luminance setting value B1 of the backlight 15b according to the acquired luminance value of the subject image and the external light illuminance L1. Here, the target luminance setting value B1 is corrected so as to be brighter when the luminance value of the subject is brighter than the external light illuminance L1.

  (S508) The control unit 4 compares the current luminance setting value B0 with the calculated target luminance setting value B1, and if the luminance difference is equal to or greater than the predetermined luminance threshold Bx, the control unit 4 proceeds to S509. If the luminance difference is less than the luminance threshold Bx, the luminance of the backlight 15b is not changed, and the process proceeds to S515.

  (S509) The control unit 4 compares the current luminance setting value B0 with the calculated target luminance setting value B1, and if the luminance difference is less than the predetermined luminance change limit value Blim, changes to the target luminance setting value B1. Even if it is determined that the user does not feel uncomfortable, the process proceeds to S510. If the brightness difference is equal to or greater than the brightness change limit value Blim, it is determined that the user feels uncomfortable when the target brightness setting value B1 is changed, and the process proceeds to S512 to limit the brightness change amount. To do.

  (S510) The control means 4 changes the brightness setting value Bs to the target brightness setting value B1, and proceeds to S511.

  (S511) The control unit 4 changes the luminance of the backlight 15b based on the luminance setting value Bs.

  (S512) The control means 4 determines whether the target brightness setting value B1 is brighter or darker than the current brightness setting value B0 in order to calculate a brightness setting value with a limited amount of change. It is determined whether or not the subtraction result between the current luminance setting value B0 and the target luminance setting value B1 takes a positive value. If it takes a positive value, the process proceeds to S513, and if it does not take a positive value. , The process proceeds to S514.

  (S513) The control means 4 changes the brightness setting value Bs of the backlight 15b to a value obtained by subtracting the brightness change limit value Blim from the current brightness setting value B0 in order to determine the brightness set value with the change amount limited. The process proceeds to S511.

  (S514) The control means 4 changes the brightness setting value Bs of the backlight 15b to a value obtained by adding the brightness change limit value Blim to the current brightness setting value B0 in order to determine the brightness setting value with the change amount limited. The process proceeds to S511.

  (S515) The control unit 4 determines whether to end or continue the display operation of the display unit 15, and when the display operation ends, the process proceeds to S516. If it is determined that the display operation is to be continued, the process proceeds to S517.

  (S516) The control unit 4 turns off the backlight 15b, performs the termination process of the display unit 15, and ends the luminance control sequence.

  (S517) The control means 4 waits for a predetermined time, returns to S501, and restarts the luminance control sequence of the display means. Here, in the same way as in the first embodiment, when the luminance difference between the current luminance setting value B0 and the target luminance setting value B1 is equal to or larger than the luminance change limit value Blim, stepwise with a waiting time of S517. To the target brightness setting value B1 corresponding to the ambient light illuminance L1.

  As described above, according to the present embodiment, since the camera attitude detection result is used, it is not necessary to change the luminance of the backlight due to a change in the external light illuminance during the change of the attitude of the display means. In addition, since the user's line-of-sight detection results and subject brightness information are used, the brightness of the backlight can be determined in consideration of the brightness even when the user's eyes are used to the brightness of the subject. When the line of sight is returned to the display means, it is displayed with a luminance that is easy to see, and it is possible to provide a display means with good visibility.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

DESCRIPTION OF SYMBOLS 1 Camera body 4 Control means 8 Image pick-up element 15 Display means 15b Backlight 16 for liquid crystals Illuminance sensor 17 Eye-gaze detection means 18 Attitude detection means

Claims (6)

Display means 15 for displaying an image, external light illuminance detection means 16 for detecting external light illuminance applied to the display means 15,
A control unit 4 for calculating a target luminance value corresponding to the detected external light illuminance and controlling the luminance of the display unit 15 so as to be a luminance value set based on the acquired target luminance value;
Line-of-sight detection means 17 for detecting the direction of the line of sight of the user,
A display control apparatus characterized by controlling the luminance of the display means 15 based on the detection result of the line-of-sight detection means 17. The control means 4 calculates the target luminance value based on the external light illuminance detected by the external light illuminance detection means 16 at predetermined time intervals,
According to the detection result of the line-of-sight detection means 16,
If the line of sight is deviated from the display means 15, the luminance value is set without limiting the amount of change of the target luminance value,
The display control apparatus according to claim 1, wherein when it is determined that the line of sight is not deviated from the display unit 15, a luminance value is set by limiting a change amount of the target luminance value.   The display control apparatus according to claim 2, wherein the control unit 4 changes the luminance value at predetermined time intervals until the target luminance value is reached. 4. The control unit 4 according to claim 1, wherein the control unit 4 changes the brightness when the detection result of the line-of-sight detection unit 16 determines that the line of sight has been deviated for a predetermined time or more. The display control apparatus according to claim 1. A posture detecting means 17 for detecting the posture or movement of the display device;
When the posture detection result by the posture detection unit 17 changes, the control unit 4
The display control apparatus according to claim 1, wherein the luminance is not changed regardless of a line-of-sight detection result and an external light illuminance detection result. A subject brightness detecting means 8 capable of detecting the brightness of the subject;
When the control unit 4 determines that the detection result of the line-of-sight detection unit 16 is out of line of sight from the display unit 15 and is directed toward the subject, the control unit 4 determines that the subject luminance detection result of the subject luminance detection unit 8 The imaging apparatus comprising the display device according to claim 1, wherein the target luminance value is corrected accordingly.