JP2011138449A - Display control device, display device, electronic apparatus and display control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically control the direction of a display image on a screen display part in accordance with a change in the direction of a user's face while preventing an increase in the operation frequency and power consumption of a camera. <P>SOLUTION: A display control device 1 includes a camera operation instruction part 2 and a screen control part 3. The camera operation instruction part 2 has the function of outputting an operation instruction signal to the camera for imaging a predetermined area facing the controlled screen display part when an inclination detection signal indicating an inclination of the screen display part is applied. The screen control part 3 has the function of controlling the direction of an image displayed on the screen display part in accordance with a predetermined detection direction of a face image captured by the camera on receipt of the operation instruction signal. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display control device, an electronic device, and a display control method for controlling screen display.

  For example, the orientation of a mobile terminal device such as a mobile phone can be easily changed. Therefore, the relationship between the vertical direction (or horizontal direction) of the screen display unit provided in the mobile terminal device and the vertical direction of the user's face looking at the screen display unit (the screen display unit and the orientation of the user) Relationship) may change. If the orientation of the display image on the screen display unit does not change despite the change in the relationship between the orientation of the screen display unit and the user, the display image is difficult for the user to see. Therefore, in Patent Document 1, the following display image switching is performed in the mobile terminal device.

  That is, in Patent Document 1, a camera provided on the same side as the screen display unit (display unit) captures the face of the user who is looking at the screen display unit every predetermined time. Then, the eyes, nose, mouth and the like are extracted by image processing of the face image data obtained by this photographing, and the vertical direction of the user's face image is determined. Further, the orientation of the display image on the screen display unit is controlled so that the determined vertical direction of the face image matches the vertical direction of characters and the like displayed on the screen display unit. In this way, by controlling the orientation of the display image on the screen display portion, even if the relationship between the orientation of the screen display portion and the user changes, a display image in a direction that is easy for the user to view is displayed on the screen display portion. It is.

JP 2008-177819 A JP 2007-89183 A

  However, in the display control method disclosed in Patent Document 1, a user's face is photographed with a camera every predetermined time. In other words, even if the relationship between the orientation of the mobile terminal device and the user has not changed over a long period of time and the orientation of the display image on the screen display unit does not need to be changed, The user's face is shot. For this reason, there are a problem that the camera has to be frequently operated, a problem of an increase in power consumption due to such a camera operation, and the like.

  The present invention has been made to solve the above problems. That is, the object of the present invention is to automatically control the orientation of the display image on the screen display unit according to the change in the orientation of the user with respect to the screen display unit while preventing an increase in camera operation frequency, camera operating time, and power consumption. An object of the present invention is to provide a display control device and the like that can be used.

The display control device of the present invention
A camera operation instruction unit that outputs an operation instruction signal toward a camera that captures a predetermined area facing the screen display unit when an inclination detection signal that informs the inclination of the screen display unit to be controlled is added;
A screen control unit that controls the orientation of an image displayed on the screen display unit in accordance with a predetermined detection direction of a face image captured by the camera in response to the operation instruction signal;
It has.

The display device of the present invention includes:
The display control device;
A screen display for displaying images;
It has.

The electronic device of the present invention is
The display device;
A camera for photographing a predetermined area facing the screen display unit provided in the display device;
Tilt detection means for outputting a tilt detection signal to a display control device provided in the display device when the tilt of the screen display unit is detected;
It has.

The display control method of the present invention includes:
When an inclination detection signal that informs the inclination of the screen display unit to be controlled is added, an operation instruction signal is output toward a camera that captures a predetermined area facing the screen display unit,
The direction of the image displayed on the screen display unit is controlled in accordance with a predetermined detection direction of the face image taken by the camera in response to the operation instruction signal.

  According to the present invention, the orientation of the display image on the screen display unit is automatically controlled according to the change in the orientation of the user's face relative to the screen display unit while preventing an increase in camera operation frequency, camera operation time, and power consumption. it can.

It is a figure for demonstrating 1st Embodiment. It is a block diagram for demonstrating 2nd Embodiment. It is a model figure which shows the example of arrangement | positioning of the screen display part and camera provided in an electronic device. It is the figure showing the state where the electronic device and the user are facing, and the example of the picked-up image of the camera in the said state. FIG. 6 is a diagram illustrating a state different from the above in which the electronic device and the user face each other, and a photographed image example of the camera in the state. FIG. 5 is a diagram illustrating a state in which the electronic device and the user are facing each other and a state different from the above, and a captured image example of the camera in the state. FIG. 6 is a diagram illustrating a state in which the electronic device faces a user and a state different from the above, and an example of a captured image of the camera in the state. It is a figure showing the example of control of the direction of the image displayed on a screen display part. It is a flowchart figure for demonstrating the example of display control operation | movement in 2nd Embodiment.

  Embodiments according to the present invention will be described below with reference to the drawings.

(First embodiment)
As shown in FIG. 1A, the display control device 1 according to the first embodiment includes a camera operation instruction unit 2 and a screen control unit 3. The camera operation instruction unit 2 outputs an operation instruction signal toward a camera that captures a predetermined area facing the screen display unit when an inclination detection signal for informing the inclination of the screen display unit to be controlled is added. It has a function. The screen control unit 3 has a function of controlling the orientation of an image displayed on the screen display unit according to a predetermined detection direction of a face image photographed by the camera in response to the operation instruction signal.

  Such a display control apparatus 1 performs the following display control operation. That is, when an inclination detection signal is applied (step S1 in FIG. 1B), an operation instruction signal is output toward a camera that captures a predetermined area facing the screen display unit to be controlled (step S2). ).

  Thereafter, the orientation of the image displayed on the screen display unit is controlled in accordance with a predetermined detection direction of the face image taken by the camera in response to the operation instruction signal (step S3).

  In the display control apparatus 1 of the first embodiment, as described above, the orientation of the display image on the screen display unit is controlled according to the detection direction of the face image obtained from the captured image of the camera. That is, the orientation of the display image on the screen display unit is controlled by detecting the orientation of the user with respect to the screen display unit from the captured image of the camera. For this reason, even if the orientation of the user with respect to the screen display unit changes, the orientation of the display screen of the screen display unit automatically changes according to the change in orientation, so that the user can continue to watch the display screen without a sense of incongruity. .

  Moreover, in the first embodiment, during the period when the screen is displayed on the screen display unit, a tilt detection signal is added instead of periodically monitoring the face image by causing the camera to perform a shooting operation. Sometimes the camera is activated to take a face image. In other words, when the orientation of the user with respect to the screen display unit changes, it is assumed that the inclination of the device including the screen display unit changes, and at this time, the camera is driven in a point manner. Thereby, the operating time and operating frequency of the camera for detecting the orientation of the user's face relative to the screen display unit can be greatly reduced without missing a change in the user's orientation relative to the screen display unit. Moreover, since the operation time of the camera can be reduced as described above, an increase in power consumption can be suppressed.

  The display control device 1 according to the first embodiment can configure a display device 5 together with a screen display unit 6 that displays an image, for example, as shown in FIG. Further, the display device 5 can constitute an electronic device 8 as shown in FIG. The electronic device 8 includes the display device 5 and also includes a camera 10 and an inclination detection unit 11. The camera 10 has a function of photographing a predetermined area facing the screen display unit 6. The tilt detection unit 11 has a function of outputting a tilt detection signal to the display control device 1 provided in the display device 5 when the tilt of the electronic device 8 is detected.

  Since the display device 5 and the electronic device 8 as described above include the display control device 1 of the first embodiment, as described above, when the orientation of the user with respect to the screen display unit 6 changes, The orientation of the display image on the screen display unit 6 can be automatically changed according to the change. In addition, it is possible to obtain an effect that it is possible to prevent an increase in operating time and power consumption of the camera 10 without missing a change in the orientation of the user with respect to the screen display unit 6.

(Second Embodiment)
The second embodiment will be described below.

  The electronic device according to the second embodiment is a mobile terminal device such as a mobile phone or a mobile information terminal device. As shown in FIG. 2, the mobile terminal device 20 of the second embodiment includes a display device 21, a camera 22, and an inclination detection unit 23.

  The display device 21 includes a screen display unit 25 and a display control device 26. The display control device 26 includes an image processing unit 28, a camera operation instruction unit 29, and a screen control unit 30.

  That is, the screen display unit 25 is configured by, for example, a liquid crystal display or the like having a function of displaying an image, and is provided on one main surface of the mobile terminal device 20 as shown in FIG. 3, for example. Examples of images displayed on the screen display unit 25 include, for example, an image of information related to a telephone function such as incoming and outgoing calls, an image of mail content of an electronic mail function, an image of Internet browsing information, a television There are video images. Furthermore, examples of images displayed on the screen display unit 25 include a preview image of the camera 22 and a photograph image.

  As shown in FIG. 3, the camera 22 is provided on the same side as the side on which the screen display unit 25 is provided in the information terminal device 20. The camera 22 has a function of photographing a predetermined area facing the screen display unit 25. More specifically, the camera 22 has a function of photographing the face of a person (user) who is looking at the screen of the screen display unit 25. Furthermore, the camera 22 has a function of automatically photographing the predetermined area using, for example, an autofocus technique when an operation instruction signal is applied from the camera operation instruction unit 29.

  The inclination detection means 23 has a function of detecting a change in the attitude of the mobile terminal device 20 (in other words, the inclination of the screen display unit 25), and includes, for example, a sensor such as a gyro as an inclination detection sensor. Yes. The inclination detection means 23 displays an inclination detection signal as a display control device assuming that the attitude of the mobile terminal device 20 has changed (the screen display unit 25 has been inclined) when the output value of the inclination detection sensor fluctuates by a predetermined value or more. 26 is provided with a function of outputting to the terminal 26. Note that the tilt detection signal is output when the mobile terminal device 20 is performing a normal operation, and is not performed when the mobile terminal device 20 is in a sleep state.

  The camera operation instruction unit 29 of the display control device 26 has a function of outputting an operation instruction signal toward the camera 22 in order to drive the camera 22 when detecting that the tilt detection signal is output. Furthermore, the camera operation instruction unit 29 is provided with a function of outputting an operation instruction signal toward the camera 22 even when a power-on signal is applied when the information terminal device 20 is turned on. Yes. Further, the camera operation instructing unit 29 is provided when the information terminal device 20 receives a hibernation state cancellation signal for shifting from the hibernation state (sleep state) to the normal state, for example, from a control unit (not shown) of the information terminal device 20. Is also provided with a function of outputting an operation instruction signal to the camera 22. Further, the camera operation instruction unit 29 has a function of outputting a camera operation notification signal for informing that the operation instruction signal has been output to the image processing unit 28 and the screen control unit 30 as described above.

  When detecting that the camera operation notification signal has been output, the image processing unit 28 has a function of capturing an image captured by the camera 22 and processing the captured image as follows. The image processing is processing for extracting the contour of the captured image taken by, for example, an edge detection processing method. Further, the image processing unit 28 has a function of determining whether or not there is a face image in the photographed image based on the extracted contour information and face contour pattern information given in advance. . Further, the image processing unit 28 has a function of outputting a face image absence signal to the screen control unit 30 when it is determined that there is no face image in the captured image.

  Furthermore, when it is determined that a face image is present in the captured image, the image processing unit 28 has a function of analyzing a predetermined detection direction of the face image. In the second embodiment, the image processing unit 28, for example, from the chin based on the contour information extracted as described above and the arrangement information of facial parts such as eyes, nose and mouth provided in advance. The upward direction of the face that goes to the forehead through the nose is calculated as the detection direction.

  For example, as shown in FIG. 4A, the information is captured by the camera 22 when the user 33 is looking at the display screen of the screen display unit 25 with the information terminal device 20 facing vertically (the camera 22 is on the upper side). Assume that the image including the face image of the user 33 is, for example, as shown in FIG. In this case, the upward direction (detection direction) of the face image 33 ′ in the captured image 34 is analyzed by the image processing unit 28 as an arrow direction along the dotted line L shown in FIG.

  Further, as shown in FIG. 5A, the image was taken by the camera 22 when the user 33 was looking at the display screen of the screen display unit 25 with the information terminal device 20 facing sideways (the camera 22 was on the right side). Assume that the image including the face image of the user 33 is, for example, as shown in FIG. In this case, the image processor 28 analyzes that the upward direction (detection direction) of the face image 33 ′ in the captured image 35 is an arrow direction along the dotted line L shown in FIG. Further, even when the information terminal device 20 is oriented vertically as shown in FIG. 6A with respect to the user 33 (the camera 22 is on the lower side), the information terminal device 20 is oriented horizontally as shown in FIG. 7A (the camera 22 is on the left side). In this case, the upward direction L of the face image 33 ′ is analyzed as described above.

  The image processing unit 28 further has a function of outputting information on the detection direction obtained by the above analysis to the screen control unit 30. For example, the inclination angle information of the detection direction L with respect to the predetermined upward direction U of the captured images 34, 35, 36, and 37 is output to the screen control unit 30 as information on the detection direction of the face image. In other words, the information on the detection direction of the face image is 0 ° in the case of FIG. 4B, −90 ° in the case of FIG. 5B, and 180 ° in the case of FIG. 6B. In the case of FIG. 7B, 90 ° is output from the image processing unit 28 to the screen control unit 30, respectively.

  The screen control unit 30 has a preview display function for displaying a preview image captured by the camera 22 on the screen display unit 25 during the operation of the camera 22. On the other hand, when receiving the camera operation notification signal from the camera operation instruction unit 29, the screen control unit 30 blocks the preview display function during the operation of the camera 22 based on the operation instruction signal (cancel). Yes) function. With this function, even when the camera 22 is operated based on the operation instruction signal, the display screen of the screen display unit 25 is not switched to the preview image of the camera 22, so that the user 33 does not have to notice the operation of the camera 22. .

  Further, when receiving information on the detection direction of the face image from the image processing unit 28, the screen control unit 30 controls the orientation of the image displayed on the screen display unit 25 in accordance with the detection direction (upward direction) of the face image. It has a function. That is, the screen control unit 30 rotates and changes the direction of the display image on the screen display unit 25 in a direction in which the upward direction of the image displayed on the screen display unit 25 matches the upward direction of the face image. Specifically, for example, when an image as shown in FIG. 8A is displayed on the screen display unit 25, information on the detection direction of the face image is transferred from the image processing unit 28 to the screen control unit 30. Is added. In this case, if the upward direction L of the face image is the case of FIG. 4B (in the case of 0 ° direction), the upward direction T of the display image on the screen display unit 25 is also the 0 ° direction. The unit 30 continues the display in the same direction without changing the direction of the display image. If the upward direction L of the face image is the case of FIG. 5B (in the case of −90 ° direction), the screen control is performed so that the upward direction T of the display image of the screen display unit 25 is directed to the −90 ° direction. The unit 30 rotates the display image as shown in FIG. If the upward direction L of the face image is the case of FIG. 6B (in the case of the 180 ° direction), the screen control unit 30 so that the upward direction T of the display image of the screen display unit 25 is directed in the 180 ° direction. Changes the display image (upside down) as shown in FIG. Assuming that the upward direction L of the face image is the case of FIG. 7B (in the case of the 90 ° direction), the screen control unit 30 so that the upward direction T of the display image of the screen display unit 25 is directed to the 90 ° direction. Changes the display image as shown in FIG.

  The mobile terminal device 20 of the second embodiment is configured as described above. Next, an example of the display control operation of the display control device 26 in the mobile terminal device 20 of the second embodiment will be described using the flowchart of FIG.

  For example, when the camera operation instruction unit 29 detects the input of any signal (start signal) of the tilt detection signal, the power-on signal, and the hibernation state cancellation signal (step S101), the camera operation instruction unit 29 outputs the operation instruction signal toward the camera 22 (Step S102). At this time, the camera operation instruction unit 29 outputs a camera operation notification signal to the image processing unit 28 and the screen control unit 30. Thereafter, the image processing unit 28 captures an image photographed by the photographing operation of the camera 22 according to the operation instruction signal (step S103). Then, the image processing unit 28 performs image processing on the captured image captured by the edge detection process or the like as described above (step S104). Further, the image processing unit 28 determines whether or not there is a face image in the captured image based on the image after the image processing (step S105). When the image processing unit 28 determines that there is no face image, the image processing unit 28 outputs a face image absence signal to the screen control unit 30. When the face image absence signal is input, the screen control unit 30 keeps the orientation of the image displayed on the screen display unit 25 (step S109).

  If the image processing unit 28 determines that there is a face image in the captured image, it further analyzes the upward direction (detection direction) of the extracted face image (step S106). Thereafter, the image processing unit 28 outputs information on the obtained face image in the upward direction L to the screen control unit 30. Upon receiving the information about the upward direction L of the face image, the screen control unit 30 determines whether the upward direction L and the upward direction T of the image displayed on the screen display unit 25 are inconsistent. (Step S107). If the screen control unit 30 determines that there is no mismatch, the image display is continued with the display screen orientation unchanged (step S109). When the screen control unit 30 determines that the upward direction L of the face image does not match the upward direction T of the display image, the direction in which the upward direction T of the display image matches the upward direction L of the face image Then, the orientation of the display image on the screen display unit 25 is changed (step S108).

  In the second embodiment, since the display control of the screen display unit 25 is performed as described above, the orientation of the display image is automatically controlled according to the orientation of the user looking at the screen display unit 25. As a result, the user can view the display image without a sense of incongruity whether the portable terminal device 20 is oriented vertically or horizontally. In addition, in this second embodiment, the user's orientation is not continuously monitored, but when the inclination of the mobile terminal device 20 is detected, when the power is turned on, and when the hibernation state is released, Detect orientation. That is, the user's orientation is detected by operating the camera 22 only at a point-like timing where it is assumed that it is necessary to detect that the orientation of the user with respect to the screen display unit 25 is changed. For this reason, compared with the case where the camera 22 is operated to continuously monitor the direction of the user, the operation time of the camera 22 can be significantly reduced. Thereby, the power consumption of the portable terminal device 20 can also be suppressed. Thereby, the battery consumption of the mobile terminal device 20 can be reduced.

(Other embodiments)
In addition, this invention is not limited to the said 1st or 2nd embodiment, Various embodiment can be taken. For example, in the second embodiment, when the face image absence signal is output from the image processing unit 28, the control operation relating to the orientation of the display image is finished as it is. Instead of this, when a face image absence signal is output, the following control may be performed. That is, the image processing unit 28 has a function of adding a face image absence signal to the screen control unit 30 and outputting the signal to the camera operation instruction unit 29. Further, when the camera operation instruction unit 29 receives the face image absence signal, the camera operation instruction unit 29 sends the operation instruction signal to the camera 22 at a predetermined timing (for example, when a predetermined time elapses after receiving the face image absence signal). Equipped with a face redetection function to output again. The camera operation notification signal is also applied to the image processing unit 28 and the screen control unit 30 when the operation instruction signal is output. By providing such a face redetection function, the following effects can be obtained. That is, even though the orientation of the user with respect to the screen display unit 25 has changed, the user has not entered the field of view of the camera 22 at the time of shooting immediately after detecting the tilt of the mobile terminal device 20, and thus the change in the orientation of the user is detected. The situation that it is not possible can be suppressed. When the screen control unit 30 determines that the face image is not extracted even if the face re-detection function is repeatedly performed a predetermined number of times based on the output state of the camera operation notification signal, the orientation of the display image May be provided with a function for terminating the control operation.

  Further, in the second embodiment, not only when the inclination of the mobile terminal device 20 is detected, but also when the power is turned on and when the hibernation state is released, the user orientation is detected based on the camera photographed image. Alternatively, the user orientation may not be detected in one or both of when the power is turned on and when the hibernation state is canceled. When the user orientation is not detected as described above, when the power is turned on or the hibernation state is canceled, for example, the relationship between the screen display unit 25 and the user is assumed to be as shown in FIG. The screen control unit 30 controls the orientation of the display image.

  Furthermore, in the second embodiment, the upward direction L of the face image is set as the detection direction. Instead of this, for example, another direction such as a rightward direction of the face image (a direction from the left eye to the right eye) may be used as the detection direction.

  Furthermore, in the second embodiment, the image processing unit 28 is provided in the display control device 26. Instead, the image processing unit 28 is not included in the display control device 26, but the mobile terminal device 20 is included. It is good also as a structure provided in the control part.

  Furthermore, in the second embodiment, the portable terminal device has been described as an example. However, the present invention can be applied to a photographing device such as a digital camera, for example, and the same effect as described above can be obtained.

1,26 Display control device 2,29 Camera operation instruction unit 3,30 Screen control unit 5,21 Display device 6,25 Screen display unit 10,22 Camera 11,23 Tilt detection means

Claims (9)

A camera operation instruction unit that outputs an operation instruction signal toward a camera that captures a predetermined area facing the screen display unit when an inclination detection signal that informs the inclination of the screen display unit to be controlled is added;
A screen control unit that controls the orientation of an image displayed on the screen display unit in accordance with a predetermined detection direction of a face image captured by the camera in response to the operation instruction signal;
A display control device.   The camera operation instruction unit further outputs the operation instruction signal toward the camera in at least one of a case where a power-on signal is applied and a case where a hibernation state cancellation signal is applied. The display control apparatus according to claim 1.   The display control apparatus according to claim 1, further comprising: an image processing unit that performs image processing on a face image captured by the camera and analyzes the detection direction of the face image. The image processing unit further receives the operation instruction signal, determines whether there is a face image in the image captured by the camera, and outputs a face image no signal when determining that there is no face image. With the ability to
The display control device according to claim 3, wherein the camera operation instructing unit further has a function of outputting the operation instruction signal again at a predetermined timing when receiving the face image absence signal.   5. The screen control unit according to claim 1, further comprising a function of blocking display of a predetermined display screen related to the operation of the camera during the operation of the camera based on the operation instruction signal. The display control apparatus according to any one of the above. A display control device according to any one of claims 1 to 5,
A screen display for displaying images;
A display device comprising: A display device according to claim 6;
A camera for photographing a predetermined area facing the screen display unit provided in the display device;
Tilt detection means for outputting a tilt detection signal to a display control device provided in the display device when the tilt of the screen display unit is detected;
An electronic device comprising: When an inclination detection signal that informs the inclination of the screen display unit to be controlled is added, an operation instruction signal is output toward a camera that captures a predetermined area facing the screen display unit,
A display control method for controlling a direction of an image displayed on the screen display unit in accordance with a predetermined detection direction of a face image photographed by the camera in response to the operation instruction signal.   9. The display according to claim 8, further comprising outputting the operation instruction signal to the camera in at least one of a case where a power-on signal is applied and a case where a hibernation state cancellation signal is applied. Control method.

Images